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Abstract 


This thesis is concerned with the quantitative assessment of security in software. 
More specifically, it tackles the problem of efficient computation of channel capacity , 
the maximum amount of confidential information leaked by software, measured in 
Shannon entropy or Renyi’s min-entropy. 

Most approaches to computing channel capacity are either efficient and return only 
(possibly very loose) upper bounds, or alternatively are inefficient but precise; few 
target realistic programs. In this thesis, we present a novel approach to the problem 
by reducing it to a model counting problem on first-order logic, which we name Model 
Counting Modulo Theories or ^SMT for brevity. 

For quantitative security, our contribution is twofold. First, on the theoretical side we 
establish the connections between measuring confidentiality leaks and fundamental 
verification algorithms like Symbolic Execution, SMT solvers and DPLL. Second, 
exploiting these connections, we develop novel ^SMT-based techniques to compute 
channel capacity, which achieve both accuracy and efficiency. These techniques are 
scalable to real-world programs, and illustrative case studies include C programs from 
Linux kernel, a Java program from a European project and anonymity protocols. 

For formal verification, our contribution is also twofold. First, we introduce and 
study a new research problem, namely ^SMT, which has other potential applications 
beyond computing channel capacity, such as returning multiple-counterexamples for 
Bounded Model Checking or automated test generation. Second, we propose an 
alternative approach for Bounded Model Checking using classical Symbolic Execution, 
which can be parallelised to leverage modern multi-core and distributed architecture. 

For software engineering, our first contribution is to demonstrate the correspondence 
between the algorithm of Symbolic Execution and the DPLL(T) algorithm used in 
state-of-the-art SMT solvers. This correspondence could be leveraged to improve 
Symbolic Execution for automated test generation. Finally, we show the relation 
between computing channel capacity and reliability analysis in software. 
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Chapter 1 


Introduction 


1.1 Motivation 

The year 2014 has witnesses several high-profile software security incidents, e.g. the 
disclosure of heartbleed bug [3], the leaks of celebrity photos in iCloud [TJ, the hack 
of Sony Pictures [9j|. The damage caused by leaking confidential information varies, 
from personal embarrassment to the lost of dozens of millions of dollars. 

These incidents show the importance of protecting confidential data from being leaked 
by software. Access control systems ra can limit access to information, but cannot 
control internal information propagation once accessed. This motivates the research 
on information flow security HD23, which aims to track the flows of information in 
software, and forbid illegal flows that leaks information to public observers. 

However, leakage of information is hardly avoidable in computer programs. Even 
“secure” programs do leak some information about the secret data being processed. 
A popular example is the password checking program, which can be considered as 
secure with a reasonably strong password. Every time it rejects an input string, it 
reveals to the adversary that the password is different from that string. This amount 
of information is small, but if the adversary is allowed to make enough attempts, i.e. 
brute force attack, the program will eventually leak all information to the attacker. 
As leakage of information is unavoidable, it is important to assess the leaks to decide 
if they are acceptable. This leads to the question how much information a program 
could leak to an adversary. 

The research area of Quantitative Information Flow analysis (QIF [43] [SU]) has been 
developed to provide a rigorous framework to answer the question above. Intuitively, 
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after observing an execution of the program, the adversary gains more information 
and has less uncertainty about the confidential data. The difference of uncertainties 
before and after his observation is the amount of information leaked by the program. 
QIF has based its foundation on the entropy concept of Information Theory |3E]: 
the uncertainties about the confidential data are measured in bits by, for example, 
Shannon entropy; leakage is then computed by numerical subtraction. 

Manual computation of QIF, e.g. in [SO], is tedious, expensive and infeasible for 
complex programs. In order to apply the theory of QIF to practice, it is crucial to 
have automated techniques that can efficiently quantify leakage in software systems. 
However, most approaches to automation of QIF are either efficient and return only 
(possibly very loose) upper bounds [S3] ED) EE], or alternatively are inefficient but 
precise [I6[ EH [75]; few target realistic programs. This thesis is a significant step 
towards efficient and accurate computation of QIF by means of formal methods. 

1.2 State of the art 

This section outlines some of the key advances that have led to the current state of 
the art for automation of QIF. Emphasis is given to work on deterministic programs. 

The concept of information flow was first introduced in the 1977 paper of the the 
Dennings [53], who described a lattice model where variables are partitioned into 
security labels: H, standing for “high”, for variables containing sensitive data and L, 
standing for “low", for variables containing public information. The partial order 
L < H in the lattice indicates that information flows from variables with label H to 
variables with label L are not allowed. 

In 1982, Goguen and Meseguer [63] described non-interference, a security policy for a 
general automaton framework where there are a set of state changing commands and 
a set of users. A group of users G have non-interference on another group of users G’ 
if and only if any user in the group G cannot observe the effects of commands used 
by any user in the group G’. 

In 1996, Volpano, Smith, and Irvine |114j achieved an impressive milestone by proving 
the soundness of the Dennings’ analysis using a type system, which coincides with 
the idea of non-interference. Since then, research in information flow has evolved in 
different directions. In 2003, Sabclfeld and Myers dna published an excellent survey 
of the field up to that time, citing 147 papers. 
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The first complete quantitative analysis of information flow had been developed by 
Clark, Hunt and Malacaria through a series of papers [4D, SB 32, 03] from 2002 to 
2007. The authors illustrated their analysis on a simple deterministic While language, 
and provided each command of the language a lower bound and an upper bound on 
the amount of leakage. Their treatment for loops is very pessimistic, assuming that 
all information in the looping conditions will be leaked. Malacaria [80] then showed 
a more precise treatment for looping constructs by using the partition property of 
entropy. This work is labour intensive and impossible to automate. 

In 2009, Smith |110] proposed to use Renyi min-entropy as a metric for QIF, and 
showed that the channel capacity of information flow, i.e. the maximum leakage, 
measured by this metric is equal to the logarithm of the number of possible outputs. 
This result agrees with previous observations of Malacaria and Chen [82] that channel 
capacity measured by Shannon entropy also has the same tight upper bound. 

Meanwhile, in the same year (2009), Backes, Kopf and Rybalchenko [IB] reached an 
important milestone by introducing the first automatic technique for QIF analysis 
using model checking [16] and Barvinok algorithm [23] for model counting. This 
analysis is very expensive, and only applicable to a limited class of programs. 

In 2010, Heusser and Malacaria jOhj published a paper demonstrating QIF analysis for 
programs from Linux kernel. Prior to this paper, all the work on QIF analysis were 
demonstrated with small “toy" examples. Heusser and Malacaria made assumptions 
that the program had N different possible outputs, and asserted that there did not 
exist an output different from those N outputs. Since these assumptions and assertion 
required the composition of N+l copies of the program, their analysis suffers severely 
from the state space explosion problem. 

A year later, in 2011, Meng and Smith [85] described a fast approximation technique 
to compute an upper bound on channel capacity. They infer the relations between all 
pairs of bits of the output as a propositional formula, then using a #SAT solver |T2j 
to count the model. The analysis was largely manual and was demonstrated with toy 
examples. The upper bound can be very loose if outputs are sparse and scattered. 

In summary, the problem of an automated QIF analysis is still very challenging. A 
simpler problem, called bounding QIF, is considered in [117L 66]: deciding if a program 
P leaks less than a constant q. In previous work, Yasuoka and Terauchi have proved 
that bounding QIF is not a k-safety problem for any k pm. Cerny et al. then 
proved that in the case of Shannon entropy, bounding QIF is PSPACE-complete [ 35] . 
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Therefore, QIF and bounding QIF remain a huge challenge. Available techniques are 
either inefficient or imprecise. This thesis investigates techniques that improves both 
precision and efficiency for QIF analysis. 


1.3 Contributions 

This thesis is about solving a problem, QIF, using a set of tools referred to as Formal 
Methods. Through the process of solving the problem, we also gain the insights to 
improve the tools. As a result, this thesis makes contributions both to approaches for 
automated QIF analysis using Formal Methods as well as techniques for improving 
Formal Methods. 

Contributions to Quantitative Information Flow 

This thesis makes a theoretical advance by casting the QIF problem into a variant 
of the Satisfiability Modulo Theories (SMT) problem [54]. This results in a general 
algorithm for QIF analysis based on the DPLL(T) algorithm |90| for SMT. This 
theoretical advance leads to practical advance: compared to the work of Heusser and 
Malacaria [IS], the technique proposed in this thesis dramatically reduces the time of 
analysing programs from Linux kernel, from some hours to a few seconds. 

The second theoretical contribution of this thesis is to show that classical Symbolic 
Execution [75] can be understood as a variant of the DPLL(T) algorithm. In other 
words, Symbolic Executors are SMT solvers. This view enables us to develop the 
first QIF analysis tool for Java bytecode, built on top of the Symbolic PathFinder 
symbolic execution platform. 

Our third contribution is to show the relation between Quantitative Information Flow 
and Reliability analysis |37j . which are two totally separate research areas prior to 
this thesis. This relation leads to the development of an efficient QIF technique based 
on an available Reliability analyser. 

The practical contribution of this thesis is the development of several QIF analysis 
tools for programs in both C and Java, and the experiments of these tools on real- 
world programs: C programs from the Linux kernel, a Java tax program from the 
European project HATS, and anonymity protocols. 
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Contributions to Formal Methods 


The insights we have learned from solving the QIF problem lead us to investigate 
techniques that could improve Formal Methods. Our first contribution in this thesis 
is the introduction of the Model Counting Modulo Theories (or ^SMT) problem, 
which has several potential applications beyond QIF. 

Our second contribution is: from the view of Symbolic Executors as SMT solvers, 
we propose a new methodology for Bounded Model Checking based on Symbolic 
Execution. Our methodology is naturally parallelizable, thus it can exploit modern 
multi-core machines and distributed architecture. Experimental results show that 
it outperforms the state-of-the-art Bounded Model Checker CBMC m in several 
complex case studies. 

The third contribution of this thesis is a light weight method for All-Solution SAT 
Modulo Theories (All-SMT). Unlike available approaches, our method can be used 
for any SMT problem with any ground theories, including bit vectors. 

Another contribution of this thesis is two applications of All-SMT solvers: the first 
one is to find multiple-counter examples in Bounded Model Checking; the second one 
is to combine Bounded Model Checking and All-SMT solver for automated test vector 
generation. 

1.4 Thesis structure 

This thesis can be divided in two parts: the first part includes chapters 00 and 
0 focusing on automation of QIF using Formal Methods; the second part includes 
chapters [4] and [5] focusing on improving Formal Methods. 

Chapter [£] provides necessary preliminaries on the two main concepts in this thesis: 
QIF and SMT. 

Chapter [3] introduces the ^SMT problem as a generalization of the SMT problem, 
and casts the QIF problem into ^SMT. The result is a general algorithm for QIF 
analysis based on the DPLL(T) algorithm used in SMT solvers. 

Chapter 0 shows how Symbolic Execution can be viewed as a variant of the DPLL(T) 
algorithm. This view enables one to turn a classical Symbolic Executor into a ^SMT 
solver for QIF analysis with little effort. 
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Chapter [5] presents a new methodology for Concurrent Bounded Model Checking 
using classical Symbolic Execution. The effectiveness of the methodology is illustrated 
with several complex case studies in both C and Java. 

Chapter [b| describes the relation between QIF and Reliability analysis. This relation 
is exploited to build an efficient QIF analysis tool for Java bytecode based on an 
available Reliability analysis engine. 

Chapter [?] presents two lightweight algorithms for ^SMT in a pure logic settings, 
and three applications of a ^SMT/All-SMT solver apart from QIF, namely multiple- 
counterexample analysis for Bounded Model Checking, automated test generation 
and reliability analysis. 

Chapter [5] concludes the thesis with a summary of contributions and discusses possible 
directions for future work. 
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Chapter 2 
Preliminaries 


In order to make the thesis self-contained, this chapter provides some basic notions 
and terminology about discrete probability theory, information theory and first-order 
theories. We also recall the elements and general concepts of formal methods. 

The content of this chapter is synthesized from several sources, e.g. m eh moi 
[Ml ESI EH EH EZJ. None of the results in this chapter were discovered by the author 
of this thesis ; a couple of proofs for well-known results, e.g. maximal entropy, are 
lifted almost verbatim from text books. 


2.1 Quantitative Information Flow 


Our attacker model is depicted in Figure 2.1[ The program P, characterized by a 
function /, takes confidential input H, public input L, and produces output 0. L may 
or may not be controlled by an adversary. The adversary tries to infer information 
from H by observing L and 0. 


H 


f 


Adversary 
tries to infer 
H from L and O 


O 



Figure 2.1: Attacker Model 
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2.1.1 Information Flow and Non-interference 


Information flow is the (illegal) transmission of information from a variable H to a 
variable 0 in a given process. The simplest case of information flow is explicit flow (or 
direct flow) where the whole or partial value of H is copied directly to 0, for example: 

0 = H + 3; 


There are more subtle cases, which are categorized as implicit flow (or indirect flow). 
Consider, for example, the program below, which simulates a common password 
checking procedure: 

if (H == L) 

0 = true; 

else 

0 = false; 


Figure 2.2: A password checking program 

H is the password, i.e. the confidential data; L is the public input provided by the user; 
0 is the observable output, “0 = true;” means the password is accepted. Although 
H is not directly copied to 0, there is still information flow leaked from H —> 0. This 
information is “small”, but one can reveal all information about H if he is allowed to 
make enough attempts. 

Obviously, information flow from confidential data to observable output is not de¬ 
sirable, which is the motivation of research in secure information flow. Dating back 
to the pioneering work of the Dennings in the 1970s [SB], secure information flow 
analysis has been an active research topic for the last four decades. 

A popular security policy that guarantees the absence of information flow leaks is non¬ 
interference [47, IBS] . It was introduced by Goguen and Meseguer as a security policy 
for a general automaton framework, here we give a definition of non-interference in 
language-based settings: 

Definition 1 (Non-interference) Suppose a program P takes secret input H, public 
input L and produces public output 0. P has the non-interference property if and only 
if: for all possible pairs of input vector [H \, Lf\, and [H 2 , L 2 ]: 


L l = L 2 /\0 1 = P([ffi, In]) A 0 2 = P([H 2 , L 2 ]) => 0 1 = 0 2 
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where Oi = P([Hi,Li]) denotes that Oi is the result of executing the program P with 
the input vector [Hi,Li]. 

We use the two terms "information flow " and " interference " interchangeably, since 
information flow from H to 0 means 0 is interfered by H. The non-interference policy 
guarantees the absence of interference or information flow. 


Type system approach to non-interference 


There has been a large body of work that has used type systems for validating non¬ 
interference, following the idea of Volpano, Irvine and Smith [114j . Type systems are 
fast and they support automated, compositional verification. Moreover, the analysis 
is safe, which means if a program is classified as “ secure ”, then it is actually secure, 
there are no false negatives. 


However, this approach is not extensible. Even a small modification in the information 
flow policy or in the programming language, e.g. adding a new feature, requires a 
non-trivial extension of the type system and its soundness proof. This approach also 
returns too many false positives, which means secure programs can be classified as 
“insecure”. For example, consider again the two examples with a small modification 
to make them satisfy non-interference: 


0 = H - H + 3; 


if (H 

= = L) 

0 

= false; 

else 


0 

= false; 


Typing rules would always classify programs like the above as insecure. Another 
tricky case is of programs that leak information in the intermediate states, but sanitize 
information at the end, for example: 

0 = H + 3; 

0 = 3; 


Given that the attacker can only observe the final value of the output 0, the program 
is secure. However, it would be classified as insecure by type systems. 
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Theorem proving approach to non-interference 


Another prominent approach for secure information flow is to use theorem proving, 
in which non-interference is logically formulated by self-composition PIES] , as non¬ 
interference itself is not a logical property. 

We assume a similar setting as in the case of non-interference: given a program P 
that takes secret input H, public input L and producing public output 0, we denote 
by Pi the same program as P, with all variables renamed: H as Hi, L as Li and 0 as 
Oi. Self-composition is expressed in Hoare-style framework as [22] : 

{L = Li}P;Pi{0 = 0i} (2.1) 


The Hoare triple states that if the precondition L = Li holds, then after the execution 
of P; Pi, the postcondition 0 = Oi also holds. The purpose of having the copy Pi with 
all variables renamed is to have another P to compare with P, so self-composition 
is logical formulation of non-interference. Compared to type system approach, the 
theorem proving approach is much more precise, returning no false positives. 


For example, consider again the password checking program P in Figure 2.2 the 


composition of P and its copy Pi is shown in Figure 2.3 By choosing H = LA Hi 7 ^ Li, 



Figure 2.3: Self-composition of the password checking program 


it is easy to find 
holds and 0 = Oi 
non-interference. 


a counter-example for the Hoare triple in (2.1), such that L = Li 
does not hold. Therefore, the password checking program violates 


Compared to type system approach, the theorem proving approach is much more 
precise, returning no false positives. However, it is impractical in reality, as elegantly 
put in |113] by Terauchi and Aiken: 
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“When we actually applied the self-composition approach, we found that not only are 
the existing automatic safety analysis tools not powerful enough to verify many real¬ 
istic problem instances efficiently (or at all), but also that there are strong reasons to 
believe that it is unlikely to expect any future advance.” 

Terauchi and Aiken also pointed out that the limitations of self-composition come 
from the symmetry and redundancy of the self-composed program, which lead to some 
partial-correctness conditions that hold between P and Pi. To find these conditions 
is crucial for the effectiveness of the analysis, however, finding them is in general 
impractical. Moreover, the use of an interactive theorem prover requires considerable 
user interaction and verification expertise. 

2.1.2 Information Theoretical Measurement of Interference 

The fact that the password checking program, Figure [A2j leaks information indicates 
that non-interference is over-pessimistic, and often unachievable. Moreover, there are 
situations that one needs to decide, between two programs, which one is more secure? 
Consider the following program: 


if (H 

/* N 

II 

A 

0 

= true; 

else 


0 

= false; 


Obviously this program is much less secure than the one in Figure 222 a fact that 
cannot be proved with non-interference. These examples show the need of a more 
quantitative assessment of information flow: instead of asking “ does the program 
leak information ?”, we would like to know “ how much does it leakl ” Information 
Theory mrzj provides the tools to answer this question. 


Discrete Probability Theory 

We introduce the concepts that are to be used in the construction of an information 
theoretical analysis of information flow. A thorough background and description can 
be found on standard textbook [58]. 

Definition 2 (Sample space) The sample space of an experiment, denoted by fl, 
is a countable set of all possible outcomes of that experiment. 


21 






Each outcome is a complete description of the state of the real world as a result of 
the experiment. It needs not be a number, e.g. the outcome of tossing a coin is either 
“head” or “tail”. Therefore, probability theory needs the following definition: 

Definition 3 (Random variable) A random variable is a function X : Q —> N 
(N C that associates a real number with each possible outcome in hi. 

Each element x G N is assigned a “ probability ” value, denoted by p(x) or p(X = x ), 
which is the measure of the likeliness that A" takes the value x. 

Definition 4 (Probability distribution) The probability distribution of a random 
variable X : Q —>■ N is a function p : N —> [0,1] such that: 

= 1 

x£N 


For discrete probability, the function p is also known as probability mass function. In 
this thesis, we are interested in the uniform distribution, in which all the elements 
of N have the same probability. This means every outcome of the sample space is 
equally likely to occur. 

Any subset E C Q, is refereed to as an event. The probability of an event E is defined 
as follows: 

p(E) = £>( A») 

lu£E 

It is straightforward from the definition of probability distribution that p(Tl) = 1, 
which means is an event that always occurs. At other extreme, it is easy to show 
that p(0) = 0, i.e. the empty set is an event that never occurs. 

Definition 5 (Conditional Probability) The probability of an event A given that 
another event B has occurred is defined as follows: 

urn = dXX 

Definition 6 (Joint Probability) The joint probability mass function of two dis¬ 
crete random variables X, Y is defined as: 

p(x,y) = p(X = x and Y — y) — p(y\x) p(x) = p(x\y ) p(y) 
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Information Theory 


Information Theory |107l 08J provides the mathematical foundation to reason about 
“information ” in a quantitative sense. Its main concept is “entropy”, which measures 
the uncertainty about a random variable. 

Definition 7 (Entropy) Given a random variable X : Q —>■ N with the probability 
mass function p(x). Shannon entropy is defined as: 

F(X) = - ^ P(x) log p(x) 

x&N 


The logarithm is to the base 2 by convention, hence the units are bits. We also write 
F(p) for the above quantity. 

Lemma 1 Given a random variable X : —>■ N with the probability mass function 

p{x), the entropy of X is bounded by: 

0 < F(X) < log \N\ 

where |iV| is the cardinality of the set N. 

Proof: By definition 0 < p(x) < 1 for all x G N, which leads to p{x)\ogp{x) < 0. 
As a result, the first part of the lemma F(X) > 0 holds. 

The second part of the lemma is proved by using Lagrange multiplier to find the 
maximal entropy. For simplicity, we define a system of index for all elements in N as 
follows: xi,x 2 , ■ ■ ■ ,x n (which also means \N\ = n). 

We write Pi for p(X = Xi), and maximize the function: 

n 

f(p 1 ,p 2 ,...,Pn) = - ^PrlogPi 

i=l 

subject to the condition of probability: 

n 

g(p 1 ,p 2 ,...,p n ) = = 1 

i —1 


Lagrange multipliers is used to find the maximum entropy p* = {pf ,p %,... ,p*} 
across all probability distributions p = {pi,p 2 , ■.. ,p n } of X. It is required that: 


^- (/ + A(s " 1)1 


p=p 


0 
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This equation is expanded into a system of n equations (i — 1,2,... ,n): 


d_ 

dpi 


~ ( lo S2 Pj + A ( - 1 


d =i 


\j= 1 


= 0 


Pi=Pi 


Carrying out the differentiation of these n equations results in: 


—+ log 2 p* ) + A = 0 


Since the value of p* only depends on A, they are all equal: 

* * * 

Pi = P2 = ■ ■ ■ = Pn 


Moreover, by definition of probability distribution: 


Ep* = 1 


This leads to: 


The maximal entropy is: 


F(p*) = 


Pi =P2 



1 

n 


n 

Pi log Pi 

1 


n 

V-log- — n — 1 ATI 
■ 1 n n 


This proves the second part Lemma [TJ F(X) < log | A/"|, equality holds when X has 
uniform distribution. 


Definition 8 (Joint Entropy) Given two random variables X : Ox —> Nx and 
Y : Oy —» N Y , the joint Shannon entropy is defined as follows: 


f(x,y ) = - p( x ^y) lo sp( x ,y) 

x£N X V&Ny 

where p{x,y ) is the joint probability mass function of X and Y. 

The joint entropy measures how much uncertainty there is in the two random variables 
X and Y taken together. 

Definition 9 (Conditional Entropy) Given two random variables X : Ox —> Nx 

and Y : Oy —>■ Ny, the conditional entropy of X given the knowledge ofY is defined 
as follows: 

HY IV = E p(x)F(Y\X = x ) 

xGNx 
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The definition of conditional entropy can be expanded as: 


F(Y\X) = - £ p(x ) E p{y\x) log p(y\x) 


xeN x y&Ny 



xeN x y€N Y 




xeN x y&N Y 


xeN x y&N Y 


F(X,Y)+ ^ p(x) log p(x) 


xeN x 


F(X,Y)-F(X) 


If X and Y are independent, then knowing the value of X does not change uncertainty 
about Y, which means F{Y\X) = F(Y). At the other extreme, if the value of Y is 
completely determined by the value of A", then there is no uncertainty about Y when 
already knowing X, and F(Y\X) = 0 holds. 

Lemma 2 Given two random variables X and Y, the following inequalities hold: 


F(X,Y) > max{F(A), F(Y)} 


Proof: Without loss of generality, assume that max{F(A), F(Y)} = F(X), we need 
to prove: 


F(X, Y) > F{X) ^ F(X) + F{Y\X) > F{X) <=> F(Y\X) > 0 


By definition of conditional entropy, we have F(Y\X) = Y2 p{%) F(Y\X = x). Similar 


to the proof of the first part of Lemma [lj we can show that F(Y\X = x) > 0 holds. 
As a result, F(Y\X) > 0 also holds. This proves the inequality in Lennna[2j Equality 


holds if and only if F(Y\X) = 0 holds, which means the value of Y is completely 
determined by the value of A". 

Definition 10 (Mutual Information) Given two random variables X : ilx —> N x 

and Y : fly —> Ny, the mutual information or mutual dependence between X and Y 
is defined as: 
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The mutual information can be rewritten as: 


y&N Y xeN x \ P\ ) / 

= -E E p(x, y ) logp(r) + E E P(x,y ) logp(x|r/) 

j/SAV x£N x y&N Y x£N x 

= F(X) - F(X|F) 


Measurement of leakage 


In the context of information flow analysis, we are interested in the value of variables 


H, L, and 0 as in our attacker model in Figure 2T For each variable, we denote its 
sample space and random variable as follows: X H : fl H —> N H , X L : f 1 L —> N L , and 
Xq : N 0 . 


The random variable Xh represents the a priori knowledge of the adversary about the 
secret data H. Consider, for example, the password checking program in Figure [E2| if 
the adversary already knows that the password H is “abc”, this means Qjj = {“abc”}, 
and p(Xh( u abc”)) = 1. Or suppose that Qh contains all possible strings of length up 
to 30, but the adversary knows in advance that the victim has the habit to use his 
daughter’s name, “Alice”, as password. In this case, in the probability distribution of 
X H , the value of p(X H (“Alice”)) is close to 1. The most general case is the adversary 
does not have any information about the password, which means Xh has a uniform 
distribution. 


The entropy F(Xh) measures the initial uncertainty of the adversary about H. When 
the adversary already knows that the password is “abc”, there is no uncertainty at 
all. (“abc”)) = 1 leads to F(Xh) = 0. On the other hand, if the adversary has 

no information about the password in advance, his uncertainty is maximal. 

After an execution of the program P, there is some information about H leaked via 
information flow from H to 0. As a result, the adversary learns some information, and 
reduces his uncertainty about H. The difference in his uncertainties before and after 
the observation is the amount of leakage: 

leakage = initial uncertainty — remaining uncertainty (2-2) 

The remaining uncertainty about H given the knowledge of O is, by definition, the 
conditional entropy F(Xh\Xo)- As a result, leakage is calculated as follows: 

A F (X H ) = F(X h ) - F(X h \X 0 ) = I(X H] X 0 ) 
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In the case of non-interference, 0 is independent from H, or in other words 0 is not 
interfered by H, F(Xh\Xo ) = F(X f ) holds. As a result, the leakage is A f (Xh) = 0. 

A F (X H ) = I(X h ;Xq ) is always positive. This property can be proved by using 
Jensen’s inequality. The interested reader is referred to the textbook by Cover and 
Thomas [38] for a proof in details. 


2.1.3 Problem Statement 


Programs such as the one in our attacker model in Figure 2.1 can be viewed as a 
channel, in which (confidential) information flows from the input H to the output 0. 
The channel capacity C is defined as the maximum amount of information can be 
transmitted in this channel. More formally, C is maximum mutual information of 
X H and Xq over all possible input distributions p of X H . 


Theorem 1 (Channel Capacity) Given a program P as a discrete channel from H 
to 0, the channel capacity C is computed by: 


C = log \N 0 \ 


Proof: By definition of conditional entropy, the leakage can be rewritten as: 

A F (X H ) = F(X h ) - F(X h \X 0 ) = F(X h ) - (F(X h , Xq) - F(X 0 )) 

= F(X 0 ) ~ (F(X h ) - F(X h ,X 0 )) 

From Lemma [2j we have F(Xh, Xo) > F(Xh)- This leads to: 

A F (X H ) < F(X 0 ) 

From Lemma [TJ F(X 0 ) < log |7Vo|, which means A F (X H ) < log \Nq\ holds. Equality 
holds when both of the equalities in Lemma [l] and Lemma [2] hold. Which means 
the value of 0 is completely determined by the value of H, and Xo has uniform 
distribution. 

As such, counting the number of observables is the basis of state-of-the-art QIF 
analysis, e.g. IBB:, E3 I H, 75] . and also the basis for this thesis. The channel capacity 
theorem also justifies the following: 

Definition 11 (The QIF problem) Given a program P, QIF is the problem of 
counting N, the number of possible outputs of P. 
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2.2 Logical Satisfiability Problems 


This section provides some logical concepts used throughout in this thesis. The 
interested readers are referred to ra for more details. 

2.2.1 Propositional Satisfiability 

Definition 12 (Propositional atom) A propositional atom or Boolean atom is a 
statement or assertion that must be true or false. 

Examples of Boolean atoms are: “all humans are mortal” and “program P leaks k 
bits”. Boolean atoms are the most basic building blocks of propositional formulas, 
each Boolean atoms A, is also a formula. 

Propositional formulas are constructed from Boolean atoms using logical connectives : 
not (-i), and (A), or (V), and imply (—>■). That means if p\ are p 2 are formulas, then 
— 1 (pi, <pi A p 2 , P\ V p> 2 , and p\ —> p 2 are also formulas. For example, {p^Ai Ad 2 ) —t A 3 
is a propositional formula. 

A Boolean atom Ai or its negation ->At is called a literal. We denote by Atom(p ) the 
set {Ai,A 2 , ■ ■ ■ A n } of Boolean atoms that occur in <p. The truth of a propositional 
formula (p is a function of the truth values of the Boolean atoms it contains. 

We denote by T and _L the truth values of true and false, respectively. 

Definition 13 Given a propositional formula ip, a truth assignment p of p is defined 
as a function which assigns each Boolean atom of ip a truth value: 

p : Atom(p) —> {T, _!_} 

A partial truth assignment of a formula ip is a function p : A —>• (T, _L} where A is any 
subset of Atom(p). A (partial) truth assignment p satisfies a propositional formula 
ip, denoted by p \= p , if p is evaluated to T under p. For example p : A 3 t—y _L 
satisfies the formula (~^A 1 A A 2 ) —> A 3 . 

A formula p is satisfiable if there exists a (partial) truth assignment such that p \= <p. 
If p |= p for every truth assignment p, then p is valid. Either a formula is valid or 
its negation is satisfiable. 


Definition 14 A propositional formula p is in Conjunctive Normal Form (CNF) if 
and only if it is a conjunction of disjunctions of literals: 

N Mi 

F — f \ \f hj 

i= 1 3 =1 

Any propositional formula can be converted to CNF by an algorithm with worst-case 
linear time 12%] . 

Definition 15 (The SAT problem) Given a propositional formula p in CNF, the 
Boolean Satisfiability Problem (SAT) is the problem of finding an assignment p that 
satisfies p. 

As the SAT problem is NP-complete, there is no algorithm that efficiently works on 
all instances of the problem. However, there are two main families of algorithms for 
state-of-the-art SAT solvers: DPLL [521EU and Stochastic Local Search [69]. This 
thesis focuses on the DPLL algorithm, which will be described in details later in 
chapter [4} 

2.2.2 Model Counting 

Definition 16 (The #SAT problem) Given a propositional formula (p, the Model 
Counting problem (#SAT) is the problem of counting all the solutions of the SAT 
problem. 

2.2.3 Satisfiability Modulo Theories 

We assume countable sets of variable V, function symbols T and predicate symbols V. 
A first-order logic signature is defined as a partial function E : TU? (d C N). 
Each a € A corresponds to the arity of an symbol. Obviously, a 0-ary predicate is a 
Boolean atom, and a 0-ary function symbol is called a constant. 

A E-term r is either a variable x G V or it is built by applying function symbols in 
T to E-terms, e.g. f(j \,..., r n ) where / G T and E (/) = n. For example, fix, g(x)) 
is a E-term if E(/) = 2 and E (g) = 1. 

Definition IT If T\, ... ,r n are Tj-terms, and p e V is a predicate symbol such that 
E (p) = n, then p(r 1; ... ,r n ) is a T,-atom. 
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A E-atom or its negation is called E-literal. We use the infix equality sign “=” as 
a shorthand for the equality predicate. If T\ and r 2 are E-terms, then the E-atom 
7~i = r 2 is called E-equality. -i(tl = r 2 ) or T\ ^ r 2 is called E-disequality. 

E-atoms are the most basic building blocks of E-formulas, each E-atom p(T\ ,..., t„) 
is also a E-formula. Similar to the construction of propositional formulas, E-formulas 
are constructed from E-terms which are glued together by universal quantifiers 
(V), existential quantifiers (3), and logical connectives. That means if p\ and 
p 2 are E-formulas, then Vx : p\, 3x : pi, -xpi, p\ A p 2 , p 1 V p 2 , and p\ -P p 2 are also 
E-formulas. 

A quantifier-free E-formula does not contain quantifiers; a sentence is a E-formula 
without free variables. A first-order theory is defined as follows: 

Definition 18 (First-order theory) A first-order theory T is is a set of first-order 
sentences with signature E. 

A E-structure M is a triple (|M|,E,Z) consisting of a non-empty domain |M|, a 
signature E, and an interpretation Z. The interpretation Z assigns meanings to 
symbols of E: for each function symbol / e T such that S(/) = n, f is assigned a 
n-ary function Z(/) on the domain \M\] for each predicate symbol p E V such that 
E( p ) = n, p is assigned a n-ary predicate Z(p), represented by a subset of \M\ n . For 
each variable x e V, Z(x) e \M\. 

A E-structure M is a model of the E-theory T if it satisfies all sentences in T. If a 
E-formula is satisfiable in a model of T, then it is called T-satisfiahle. 

Henceforth, for simplicity we will omit the prefix “E—” from term, atom, formula, 
etc. Instead, we will often use the prefix “T-” to denote “in the theory T”. 

We define a bijective function BA ( Boolean abstraction ) which maps Boolean atoms 
into themselves and T-atoms into fresh Boolean atoms. The Boolean refinement 
function BIZ is then defined as the inverse of BA, which means B7Z = BA V . 

Definition 19 (The SMT problem) Given a theory or a combination of theories 
T and a E- formula p, the Satisfiability Modulo Theories problem (SMT) is the prob¬ 
lem of deciding T-satisfiability of p. 

Most of the work on SMT focus on quantifier-free formulas, and decidable first-order 
theories. The SMT problem is NP-hard, as it subsumes the SAT problem. 
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Most state-of-the-art SMT solvers implement the DPLL(T) algorithm, which is the 
integration of two components: (i) an enumerator integrating a DPLL-based SAT 
solver enumerates truth assignments satisfying the Boolean abstraction of the input 
formula; (ii) T-solvers validate the consistency w.r.t. theories T of the (partial) 
assignment produced by the SAT solver. The DPLL(T) algorithm will be described 
in more details later in chapter |4| 


2.3 The programming language and the program 


For simplicity, we illustrate our methodologies using the guarded command language 


instead of C or Java. The grammar of the language is depicted in Figure 2.3 


program = stmt* 

stmt s = assume e | assert e | v = e | if e then goto s else goto s 

v G Var (variables) 

e G Exp (expressions) 

Figure 2.4: The guarded command language 

In this thesis, we focus on safety properties: note that the two commands assume(c ) 
and assert(c) are powerful enough to encode expressive temporal properties [T5|, and 
also support assume-guarantee style compositional reasoning. Any program can be 
viewed as a system that transits between states. There are many ways to describe 
this system depending on how much detail of the program that needs to be captured. 
Apart from chapter [4], in this thesis a program P is modelled as a transition system 
as follows: 

P — {S,I,F, T) (2.3) 

where S is the set of program states; / C S is the set of initial states; F C S is the 
set of final states; and T C S x S is the transition relation. 

Under this setting, a trace of (a concrete) execution of the program P is represented 
by a sequence of states: p = s 0 s i-- s fc such that s 0 G /, Sk G F and (s,, s i+ i) G T for 
all i G (0,.., k — 1}. 

We define two functions init and fin to get the initial state and final state of p: 
init(p) = So and fin(p) = sy The semantics of P is then defined as the set 7Z of all 
possible traces. 
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In the context of the information flow problem, we assume that each initial state 
s G / is a pair (H,L), i.e. I — Ih x II, in which H is the confidential component to 
be protected and L is the public component that may be controlled by an attacker. 

2.4 Formal Methods 

Formal methods refer to a set of mathematical-based techniques used in Computer 
Science for the specification and verification of software and hardware systems. These 
techniques base their foundations on several conceptual frameworks: automata theory, 
logic calculi, formal languages, program semantics and so on. 

The act of using formal methods to prove or to disprove the correctness of a system, 
with respect to a certain property, is called formal verification. Compared to testing, 
formal verification is much more expensive. However, it is crucial in the development 
of systems whose failure can cause huge financial lost or even cost human lives. History 
has witnessed several computer-related disasters that could have been prevented if 
formal verification had been used [3Ej. 

There are three main components involving in the formal verification of a hardware 
or software system: 

• A formal model of the system. Models used in formal verification vary in the 
level of abstraction, from an automaton describing status changes of the system, 
to source code or machine code of the system. 

• A formal specification, often described in a formal languages. These formal 
languages also have different power of expressiveness. 

• A formal method, implemented in a fully or partially automated tool, to prove 
or disprove the conformance of the formal model to the formal specification. 

There are three possible cases for the result: the first case is the program conforms to 
the specification; the second case is the system violates the specification, in which a 
counterexample might be returned; the final case is the tool fails to prove or disprove 
within a period of time. 

Naturally, there is a trade-off between the level of abstraction of the model and 
the expressiveness of the specification. Typically, formal methods-based tools can 
check complicated specification on highly abstract models, and simple specification 
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in detailed models. In this thesis, the formal model of a program is C source code or 
Java bytecode. The formal specification is the reachability of some assertions in the 
source code or bytecode. The formal methods we used are Bounded Model Checking 
and Symbolic Execution. 

2.4.1 Bounded Model Checking and CBMC 

The aim of Bounded Model Checking [26J (BMC) is to find bugs or to prove their 
absence up to some bounded k number of transitions. Recall that a program P is 
modelled as a transition system P = (S, I, F,T), and a trace is represented by a 
sequence of states p = SQSi-.Sk- 

A trace can be also seen in logical form: the set / and the relation T can be written 
as their characteristic functions: s 0 G / iff J(s 0 ) holds; (si,s i+ 1 ) G T iff T(si,s i+ 1 ) 
holds. In this way, a trace p is represented by the formula: 

fc-i 

I(s 0 ) A/\T(si,s i+1 ) (2.4) 

i =o 

Clearly the transition system P is a model for such a formula, i.e. P is a model 
for all formulas representing traces of the program. As BMC aims to find bugs or 
prove their absence up to some bounded k number of transitions, it explores all traces 
p = SoSi-.Sfc of the program P, in which needs not to be in F. 

Notice that because of the bound k there are only a finite number of traces to explore. 
Hence we can represent the bounded program as a formula C which is a conjunction 
of formulas, whose conjoints are possible traces. Notice formulas can also represent 
symbolic traces, for example if in a formula the value of a program variable is left 
unspecified then there can be several concrete traces satisfying that formula. Formulas 
satisfied by set of concrete traces can be referred to as symbolic traces. 

CBMC translates a C program into a logical formula C which is then used as a model 
for the property V to be verified. The property is verified by the C program iff C A V 
is valid. This can be checked by a satisfiability solver on C A -PP. In fact if C A ->V is 
true in the model then one trace will satisfy ->V hence the property is not valid. On 
the other hand if C A ~PP is false in the model then no trace will satisfy hence V 
is valid. 
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2.4.2 Symbolic Execution and Symbolic PathFinder 


Symbolic Execution [T4j (SE) is a programming analysis technique which executes 
programs on unspecified inputs, by using symbolic inputs instead of concrete data. 
For each executed program path, SE builds a path condition which is the condition on 
the inputs for the execution to follow that path, according to the branching conditions 
in the code. 

A path condition pc is initialized as empty, and it doesn’t change when executing 
non-branching instructions. For an if statement with condition c, there are three 

possible cases: (i) pc h c: SE chooses the then path; (ii) pc I- <c: SE chooses the 

else path; (iii) (pc Y- c) A (pc Y —> c): SE executes both paths: in the then path, it 
updates the path condition pc\ = pc Ac, in the else path it updates the path condition 
pc 2 = pc A —>c. 

In classical SE, the satisfiability of the path condition is checked at every branching 
point, using off-the-shelf constraint solvers. In this way only feasible program paths 
are explored. A symbolic execution tree characterizes the execution paths followed 
during the symbolic execution of a program. The nodes represent program (symbolic) 
states and the arcs represent transitions between states. 

Symbolic PathFinder (SPF) is a SE framework built on top of the Java PathFinder 
(JPF) model checking tool-set for Java bytecode analysis. It implements a bytecode 
interpreter that replaces the standard, concrete execution semantics of bytecodes with 
a non-standard symbolic execution. 

SPF implements classical SE, in its default mode SPF only explores feasible symbolic 
paths. However, it also has an option to run without constraint solving, which means 
for an if statement with condition c, both pc Y- c and pc Y- ->c are assumed to be 
true. As a result of this option, SPF will explore all the possible paths (feasible and 
infeasible) through the program, up to the given bound. This particular option will 
be used later in chapter [5] for the design of a concurrent bounded model checker. 
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Chapter 3 

Model Counting Modulo Theories 


This chapter introduces the t^SMT problem and a T^SMT-based technique for QIF 
analysis, which provides a dramatic improvement on state-of-the-art implementations 
of QIF analysis. On the theoretical side, this work establishes a connection between 
fundamental verification algorithms and QIF. This connection is exploited to mitigate 
the state explosion problem by developing a novel approach for QIF based on SMT. 
More specific contributions are: 

1. Introduction of a new research problem, Model Counting Modulo Theories or 
#SMT, and its applications to QIF. 

2. A framework, called ^DPLL(T), to build a solver for #SMT-based QIF. 

3. A prototyping tools for QIF analysis: sqifc built on top of CBMC [44] , 

4. Analysis of complex code, including recent vulnerabilities from the National 
Vulnerability Database of the US government [6] and anonymity protocols. 

3.1 Illustrative Example 

To illustrate our approach, consider the data sanitization program P from [85, 85] , 
shown in Figure [3J~| It is straightforward to show that only integer values from 8 to 
23 are possible outputs of this program. An attacker has hence available 16 possible 
output observations: observing outputs 9 .. 23 will know the secret H is 1 .. 15 and 
observing 8 will know the secret is 0 or greater than 15. Assuming the attacker has no 
prior knowledge of the secret H apart that is a 32 bits variable his a-priori probability 
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L = 8; 
if (H < 16) 

0 = H + L; 
else 

0 = L; 


Figure 3.1: Data sanitization 


of guessing the value of H in one try is and the expected probability of guessing 
the secret in one try after observing the outputs is: 

15 | 2 32 — 15 1 15 , 1 _ 16 

232 2 32 2 32 — 15 — 2 32 2 32 _ 2 32 


We can measure the leakage of the program as the difference (of the — log base 2) 
between the probability of guessing the secret before and after observing the outputs 
of the program; in this case: 

1 16 

” lQ g(^) - (- log = log(16) = 4 

The result of this measurement, log(16) = log (number of output observations), is 
an alternative explanation for theorem [lj i.e. theorem of channel capacity, that we 
have proved in the previous chapter. Our goal is to develop an efficient automated 
technique to compute this number of output observations. 


Notice that the output 0 is stored in the computer memory as a string of 32 bits 
b\b 2 ... & 32 , which can be represented by a set of Boolean variables Vj = {pi,p 2 , ■ ■ ■ P 32 } 
such that Pi = T if and only if bi is 1, and Pi = _L if and only if b, — 0. Thus, each 
possible value of 0 corresponds to a truth assignment for Vj. For example, 0 = 1000b 
corresponds to: p\ t->- _L,p 2 | —> J-,p 3 1 —> _L,p 4 1 —> T,p 5 1 —> _L,... , p 32 1 —> T. 


Since there are 16 possible values of 0 from 8 to 23, there are 16 possible truth 
assignments for Vj. We can view these truth assignments as partial models of a 
logical satisfiability problem on a logical formula tpp. Obivously, this formula tpp 
characterizes the behaviour of the program P because of the correspondence between 
a possible output of P and a partial model of p P . 

Our goal is to count all possible values of 0, which corresponds to counting all models 
of ipp with respect to the set Vj. I 11 other words, we cast the problem of measuring 
information leaks of P to a model counting problem on ipp. In the next sections, we 
will give a formal definition for this problem, which we name Model Counting Modulo 
Theories, and develop an algorithm for QIF analysis based on it. 


36 




3.2 Model Counting Modulo Theories 


In the previous chapter, we have recalled three logical satisfiability problems that 
have been studied extensively in recent years, namely SAT, ^SAT, and SMT. Their 


relations with each other are depicted in Figure 3.2 Since propositional logic is a 
special case of first-order theories whose signature contains only 0-ary predicates, the 
SAT problem is a simple case of the SMT problem. Moreover, the ^SAT problem is 
a generalization of the SAT problem from finding one model to counting all models. 
What is lacking in this big picture is a satisfiability problem that is a generalization of 
^SAT to first-order theories, and is a generalization of SMT from finding one model 
to counting all models. 


generalize to first-order theories 
SAT -> SMT 

generalize to generalize to 

counting models counting models 

generalize to first-order theories 
#SAT-> ? 

Figure 3.2: Logical satisfiability problems 

Note that it is not always possible to count models in SMT as most SMT theories 
permit an infinite number of models. For example, there are uncountably infinite 
models for the formula: ip = A A (x > 1), where the background theory T is £«4(Q), 
the theory of linear arithmetic over the rationals, which means i6Q. 

Here we restrict the problem to counting all models with respect to a set of Boolean 
variables. This restriction guarantees that there are always finite number of models 
regardless of the background theories. 

Definition 20 (The #SMT problem) Given a theory or a combination of theo¬ 
ries T and a T,-formula p>, the Model Counting Modulo Theories problem (#SMT) is 
the problem of counting all models M of T with respect to a set of Boolean variables 
Vi such that <p is T-satisfiable in M. 
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SAT 


generalize to first-order theories 


> SMT 


generalize to generalize to 

counting models counting models 

generalize to first-order theories 
#SAT-> #SMT 

Figure 3.3: Logical satisfiability problems 

With this new ^SMT problem, our picture of logical satisfiability problems in Figure 
3.3| is now complete. The ^SMT problem is a generalization of both the t^SAT 
problem and the SMT problem. 

Recall that most state-of-the-art SMT solvers are the integration of two components: 
(i) an enumerator integrating a SAT solver enumerates truth assignments satisfying 
the Boolean abstraction of the input formula; (ii) T-solvers validate the consistency 
w.r.t. theories T of the (partial) assignment produced by the SAT solver. 

Naturally, an SMT solver can be extended into a ^SMT solver by replacing the SAT 
solver with a ^SAT solver that can explicitly enumerate all models. 


3.3 Quantitative Information Flow as #SMT 


We assume the setting in our attacker model in Figure [2~T| a program P that takes 
secret input H, public input L and producing public output 0. As per definition 11, the 
quantitative information flow problem is to count the number N of possible values of 
the output 0, which is an M -bit data • • • &m- 


Assuming that we have a (first-order) formula ipp with the following properties: (i) 
c p P contains a set of Boolean variables Vj := {p 1 ,p 2 , • •, Pm }; (ii) Pi = T if and only if 
bi is 1, and Pi — T if and only if bi = 0. Under these settings, the QIF problem of 
counting N can be viewed as a ^SMT problem on the formula p>p and the set Vj of 
Boolean variables. 


Under this view, on one hand we have a program P to perform QIF analysis, and a tool 
box of formal methods. On the other hand, we have a logical formula (pp to compute 
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^SMT and the DPLL(T) algorithm. Hence, there are two possible approaches to 
our QIF problem: the first one is to construct such a formula ipp from the program 
P, then solving it with an t^SMT solver; the second one is to use formal methods in 
a way that mimics the DPLL(T) algorithm. 

P <-♦ <Pp 

QIF #SMT 

Formal methods DPLL(T) 

The first approach is more intuitive. However, it is also more complicated, since there 
is no available off-the-shelf ^SMT solver. Therefore, we will leave it for chapter [TJ 
In this chapter, we will explore the second approach, which is much simpler yet 
powerful enough to analyse real-world programs, and outperform dramatically the 
state-of-the-art technique. 

An extremely naive technique for QIF using formal methods is to check each number 
one by one if it can be a value of the output 0. This can be done as follows: 

N = 0 

for all v from 0 to 2 M do 

if (assert 0 != v is violated) 

N <- N + 1 

end for 
return N 


We make an assertion that the output 0 is always different from v, then checking the 
validity of this assertion using formal methods. If the assertion is valid, then v is not 
a possible value of 0. On the contrary, if the assertion is violated, then 0 can take the 
value v, and we increase the counter. 

Assuming that we have a very powerful tool that can verify each assertion in one 
second, the procedure would take around 2 32 seconds, which is approximately 136 
years. Therefore, this technique is impractical. The reason is that it checks one 
concrete value at a time, and thus it is vulnerable to the state-space explosion problem. 
Although this technique is naive, it inspires us to develop a procedure to process 
multiple values at a time. 

Consider again the set of Boolean variables Vj := {pi,P 2 , --,Pm}, for example the 
partial truth assignment /j = {pi i-> T,p 2 *-> -L} represents 2 A/ ~ 2 concrete values: all 
the bit configurations over M bits where the first bit is 1 and the second bit is 0. 
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Thus, if we can verify that p>p cannot be satisfiable in /j, we can ignore those 2 M ~ 2 
values. Although for this approach we do not construct the formula <pp, checking 
that ipp cannot be satisfiable in /i can be done by using formal methods to check the 
assertion that — 1(61 = 1 && b 2 = 0 ) is valid. 

So a partial truth assignments of Vj is a symbolic representation for a set of values 
of the output 0. Based on this, we develop a technique, called Symbolic Quantitative 
Information Flow that mimics the DPLL(T) algorithm and explores the state space. 
Recall that DPLL(T) algorithm consists of a DPLL-based SAT solver to enumerate 
(partial) truth assignments, and a T-solver to check the consistency of these truth 
assignment. As we use formal methods, in particular model checking, to check if the 
formula ipp can be satisfiable in a partial truth assignment //, the model checker plays 
the role of the T-solver. 

P <-> Fp 

Model Checker T-solver 


3.4 Symbolic Quantitative Information Flow 

Our first step is to construct the set of Boolean variables Vj that we have described. 
In a language that supports bitwise operators such as C/C++ and Java, this can be 
done by instrumenting the program P as follows: 


for all i from 1 to 

M do 

bi = (0 >> (i - 

i))&i 

if (bi == 1 ) 


Pi «- T 


else 


Pi -L 


end for 



Figure 3.4: Program instrumentation 


This instrumentation guarantees that the variable pi corresponds to the i th bit of the 
output 0 . A high level framework to explore the state-space and quantify the leaks 
of confidential data is described by the procedures SyrabolicQIF and SymCount in 
Figure |3.5| and [+6 
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function SYMBOLlcQIF(Vr, (pp) 
W = e, pc = e, TV = 0, i = 1 
Ear lyPrunning ( Vi ) 

SymCount( Vj , \P, <£>p, TV, pc, i ) 
return !F, log 2 (TV) 


Figure 3.5: Symbolic QIF analysis 


1 

function SymCount(Vj, &,(pp, N,pc, i ) 

2 

if (TV > 2 k ) return Insecure 

3 

Extract p t from Vj 

4 

pc.\ pc A pi 

5 

if (T-solver(</?p,pci)) 

6 

if (i == M) 

7 

$ <r~ $ U {pCi} 

8 

TV TV + 1 

9 

else 

10 

SymCount(V/, H, <pp, TV, pci, i + 1) 

11 

pc 2 pc A -ip* 

12 

if (T-solver(<^p,pc 2 )) 

13 

if (i == M) 

14 

1/ *r~ V U {pC 2 } 

15 

TV TV + 1 

16 

else 

17 

SymCount(V/, W,(p P ,N, pc 2 , i + 1) 


Figure 3.6: Symbolic counting for QIF 
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Vj, T Tp and N are passed by reference, while pc and i are passed by value. Vj 
is the symbolic representation of the output described in the previous section, ipp 
is the formula representing the program P and P is the set of models of ipp. N is 
the cardinality of P, and the procedure SymbolicQIF returns log 2 (lV) as the channel 
capacity. 


M is the size of the output data type, e.g. M = 32 if O is a 32-bit integer, and i 
is the depth of the recursive call. The parameter pc is a partial assignment of Vj, it 
is incrementally updated when the search progresses. In SymCount, T-solver(ip p, pc) 
means the T-solver is called to check if there is a model of pp where pc is (assigned 
to) T. 


We illustrate the algorithm SymCount by running it on a simple example (we ignore 
temporarily lines 2 and 3 that will be clarified in section 3.5). Consider again the case 
study of the data sanitization program in Figure 3.1| Only integer values from 8 to 
23 are possible outputs of this program, which means the number of possible outputs 
is N = 16. 


At the beginning, all variables are initialised in the procedure SymbolicQIF as in 


Figure 3.5, the method EarlyPrunning employs a heuristic that will be discussed 
later in this section. The method SymCount is then called to count the number of 
possible models of <pp. 

When a variable pi G Vi is selected, we systematically explore in the same way for 
both pi and ~'p l . Hence, the block of code from line [4] to line 10, and the one from 
line 11 to line 17 in Figure [4] are symmetric: we only explain the first one. 

A partial run of SymCount on the illustrative example is depicted in Figure |3.7 At 
the first call of SymCount: i — 1, the variable p\ is in consideration and it is added 
to pc in line [4} Since pc is initialised to be empty, pc± = p\. The T-solver is called 
to check if there is a model of ipp where pi is (assigned to) T. This can be done by 
using assertion to check the validity of ->px in a program as follows: 


assert !pi; 


A model checking tool like JPF or CBMC can be used as a T-solver to verify this 
assertion and it will return T if the assertion fails, and False otherwise. In this 
example, the T-solver would return T since p\ stands for “first bit is 1” and all odd 
values from 9 to 23 are possible outputs satisfying the condition p\. Hence, SQIF 
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Figure 3.7: Partial exploration path of SQIF for the program from Figure 3.1 


proceeds by calling SymCount with i = 2. Similarly, the procedure progresses until 
calling SymCount with i = 5, which means it needs to verify: 

assert \(p 1 && p 2 && p 3 && p 4 && p 5 ); 


This time the T-solver would return False , since pi A p 2 .. A p$ represents a set of 
outputs of which each element is at least 2° + 2 1 + .. + 2 4 = 31, while the possible 
range of O is only from 8 to 23. For a program with an output of 32-bits, by using 
EarlyPrunning, SQIF trims a set of 2 2 ' concrete values represented by the family of 
sets: 

{Vi := {pi,P2, P 32 } ■ Pi Ap 2 Ap 3 Ap 4 Ap 5 } 

This is how the state-space explosion problem is mitigated. 


At the depth i = 5 as above, if SQIF takes the path of ->p$ from line 11, then the 
T-solver returns T (0 — 15 is one of the models). Hence, the procedure continues 
with i — 6, and from this point until i = 32, only the path of ~<p t is SAT. At i — 32, 
SQIF finds a full path 00..01111 which represents an output O = 15. This path is 
added to F, and SQIF increases N. Finally, at the end of the method SymbolicQIF, 
we have F = {8, 9,.., 23} and N = 16, thus we can conclude that the data sanitization 
program in Figure |3T leaks at most 4 bits. 


The method EarlyPrunning implements the idea that if pi is unsatisfiable, then pi AC 
is also unsatisfiable for any C. Therefore, at the beginning of the SymbolicQIF, all 
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Pi are checked for satisfiability, and the results are stored for later use. We note that 
EarlyPrunning speeds up SymbolicQIF dramatically when the number of possible 
models (outputs of the program) is small. 

We have developed a prototyping tool for QIF analysis of C programs, sqifc built on 
top of CBMC. 

3.5 Soundness and Completeness 

By soundness of the SQIF approach we mean that given F, log 2 (iV) returned by 
SymbolicQIF(V/, <pp), each element of F is a model of (pp i.e. corresponds to a 
possible value of the output of the program P. By completeness of SQIF, we mean 
that F is the set of all models of <pp i.e. all values of the output of P. 

Theorem 2 Given a sound (resp. complete) T-solver the SQIF approach is sound 
(resp. complete) i.e. SymCount solves the QIF problem ( Definitional 7]). 

Proof sketch 1 The SQIF algorithm as described in Figure |ff.h| is based on DPLL 
which itself is a depth-first search procedure. As the search space is a binary tree with 
bounded depth M, the number of bits of the output, the depth-first search procedure 
is complete. The soundness of SQIF is guaranteed by the soundness of the T-solver, 
i.e. model checker. 

In reality T-solvers are only complete in particular domains. Moreover, even with 
sound and complete T-solvers, a large leak requires an exponential number of calls 
to the T-solver and so in practice SQIF is complete only for programs with small 
leaks. Since our tools are based on bounded model checker , we choose to analyse 
only bounded programs. Notice however that Theorem [2] holds for general T-solvers. 

Because of these practical issues about completeness, it has been proposed to shift 
the focus from the question “How much does it leak?" to the simpler quantitative 
question “ Does it leak more than k?" [66., 117] , This approach not only makes the 
problem easier to analysis, but it is also more intuitive in term of security, because the 
user policy, i.e. threshold k, is encoded in the analysis. The ultimate goal of security 
analysis is to determine whether a program is secure or insecure. As discussed in the 
previous section, the goal of QIF is to relax security policy from non-interference to 
an acceptable threshold k bits of interference, so that we can tolerate “ small " leak, 
and accept more programs as secure. The SQIF approach can also be used in the 
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same way: with a user policy k, if SQIF finds out more than k = 2 k possible outputs, 
we can stop the procedure and conclude that the program is insecure. This is the 
meaning of lines [2] and [2] of function SymCount in Figure [4] 

A straightforward consequence of the Theorem [2] is that, assuming a sound T-solver, 
given a user policy k, SymCount never returns secure for a program leaking more than 
k bits. This can be formally expressed as: 

Corollary 1 SQIF is sound w.r.t a user policy k. 


3.6 Evaluation 

Only few papers present QIF static code analysis of real-world applications: examples 
are esi. ra and the more recent DEI- Of these three approaches, ca uses a different 
attacker model, namely cache side-channels and so is not directly comparable with 
our approach. The other two, [66] and ESI. use the same attacker model as we do 
but are at the moment both restricted to C programs, and hence only comparable to 
sqifc. We will concentrate on [66], to which we refer as selfcomp, because it is based 
on the well-known concept of self-composition [22], For the analysis of anonymity 
protocols, we compare sqifc against QUAIL [SJI2ZJ, a state-of-the-art quantitative 
analyser for probabilistic programs. The case studies broadly fall in three categories: 

• the first category, consisting of case studies from the National Vulnerability 
Database of the US government [6], is aimed to demonstrate how our analysis 
is able to deal with complex C-code, 

• the CRC case study shows the applicability to quantify leakage in applications 
which leak by design, 

• the case studies Grade and Dining cryptos protocols show how our technique, 
even if it is unable to analyse probabilistic programs, is able to computing 
channel capacity for anonymity protocols. 

The experiments are conducted on a desktop machine with Intel Core i5 3.3GHz and 
8GB of memory. 
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3.6.1 CVE-2011-2208 


This case study is an example of a program that leaks information when the attacker 
can control the public input, ft is taken from the National Vulnerability Database 
(NVD) of the US government [6], and it is released on 13/06/2012. 


1 

int osf_getdomainname(char __user *name, int namelen) 

2 

1 

3 

unsigned len; 

4 

int i , error ; 

5 


6 

error = verify_area(VERIFY_WRITE, name, namelen); 

7 

if (error) 

8 

goto out ; 

9 


10 

len = namelen; 

11 

if (namelen > 32) 

12 

len = 32; 

13 


14 

down_read(&uts_sem) ; 

15 

for (i = 0; i < len; ++i) { 

16 

__put_user(system_utsname.domainname [i] , 

17 

name + i); 

18 

if (system.utsname.domainname [i] == ’\0 ’ ) 

19 

break; 

20 

} 

21 

up_read(&uts_sem); 

22 

out : 

23 

return error ; 

24 

1 


Figure 3.8: arch/alpha/kernel/osf_sys.c 


The system call osf_getdomainname, depicted in Figure 3.8, in the Linux kernel 


before 2.6.39.4 leaks sensitive information from kernel memory. This is caused by an 
integer signedness error: the signed parameter namelen is assigned to the unsigned 
variable len in line 10, so a negative value can be transformed into a big positive one. 
Therefore, although the condition in line 11 restricts namelen to 32, the number of 
characters returned to the user via the structure name may be much greater including 
bytes from kernel memory. 


In order to quantify the information leakage caused by this vulnerability, we chose 
the thresholds of security policy k = 64 and k = 256, which means the program is 
secure if it leaks less than 6 and 8 bits respectively. After the times in Figure [3. 11| , 
sqifc and selfcomp conclude that the program is insecure. We then apply the patch 


46 






provided for this vulnerability, and run sqifc again. This time, sqifc found only one 
possible value for name, which means a leak of zero bit. Hence, we prove that the 
patch fixed the leak. 


3.6.2 CVE-2011-1078 


This case study is also taken from NVD, and it is released on 21/06/2012. The 
function sco_sock_getsockopt_old in the Linux kernel before 2.6.39, depicted in 
Figure [T9 leaks sensitive information from kernel memory. 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 

27 

28 

29 

30 

31 

32 


static int sco_sock_getsockopt_old( 

struct socket *sock, int optname , 

char __user *optval, int __user *optlen) 

1 

struct sock *sk = sock->sk; 
struct sco.conninfo cinfo; 
int len, err = 0; 


lock.sock(sk) ; 

switch (optname) { 
case SC0_0PTI0NS : 


case SC0_C0NNINF0 : 


cinfo.hci_handle = sco_pi (sk)->conn->hcon->handle; 
memcpy(cinfo.dev_class, 

sco_pi(sk)->conn->hcon->dev_class, 3); 

len = min_t(unsigned int, len, sizeof(cinfo)); 
if (copy_to_user(optval, (char *)&cinfo, len)) 
err = -EFAULT; 
break; 

1 

release.sock(sk); 
return err ; 


Figure 3.9: net/bluetooth/sco.c 


As in line 24, cinfo is copied to the user. Although its total size is 5 bytes, and all 
bytes are correctly assigned, when compiled it includes an additional padding byte 
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for alignment purposes. This padding byte is not zeroed out, and hence it contains 
kernel memory, and is leaked to the user. Results of the analysis for k = 8 and k = 64, 


are shown in Figure 3.11 


3.6.3 Cyclic Redundancy Check 


The program in Figure 3.10 performs Cyclic Redundancy ChedsQ (CRC) and shifts 
right the result sft bits. We also have a Java version of the program to test with 

jpf-qif. 


unsigned char GetCRC8( 

unsigned char check , unsigned char ch) 

{ 

int i , sft ; 

for ( i = 0 ; i < 8 ; i++ ) { 

if ( check k 0x80 ) { 

check < < = 1; 

if ( ch k 0x80 ) { check = check I 0x01;} 

else { check =check & Oxfe; } 
check = check “ 0x85; 

} else { 

check < <=1; 

if ( ch k 0x80 ) { check = check I 0x01; } 

else { check = check k Oxfe; } 

} 

ch<<=l; 

} 

check >>= sft ; 
return check ; 


Figure 3.10; Cyclic Redundancy Check 


We quantify the amount of information of the confidential input ch revealed by ob¬ 
serving the output of function GetCRC8. We analyse this program with sqifc and 
selfcomp for sft values of 3 and 5 giving a maximum leakage for this program of 
5 (selfcomp times out on this case) and 3 bits respectively which is consistent with 


the design of the program. Results of the analysis are shown in Figure 3.11 In the 
case value of sft is 5, i.e. k = 8, selfcomp is faster as the state-space is still small 
enough, and selfcomp requires only one call to CBMC. When sft = 3, i.e. k = 32, 
the state-space explosion makes selfcomp fail to solve. SQIF requires several calls 
to the solver, but it is less vulnerable to state-space explosion. 


1 http://en.wikipedia.org/wiki/Cyclic_redundancy_check 
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Case Study 

Policy 

LoC 

sqifc time 

selfcomp time 

Data Sanitization 

- 

< 10 

11.898 

timed out 

CVE-2011-2208 

64 

> 200 

22.759 

119.117 

CVE-2011-2208 

256 


88.196 

timed out 

CVE-2011-1078 

8 

> 200 

10.380 

13.853 

CVE-2011-1078 

64 


37.899 

timed out 

CRC 

8 

< 30 

1.209 

0.498 

CRC 

32 


8.657 

timed out 


Figure 3.11: Comparing sqifc against selfcomp. Times are in seconds, timeout is 30 
minutes. In the first case study, means the policy is not specified. 


3.6.4 The Grade Protocol 


This case study was used to illustrate protocol analysis in H2JEU. This anonymity 
protocol is designed to enable a group of students to compute the sum of their 
grades (e.g., to compute the average) without revealing individual grades. We de¬ 
note Si, ...,Sk be the k students arranged in a ring, each one is given a secret grade g* 
between 0 and m — 1. To compute the sum of g % without disclosing them, the students 
produce k random numbers between 0 and n — {m — l)k + 1 such that the number 
r l is known only to the students S) and S^ + i)%fc. Each student Si then outputs a 
number di = g t + r * — T( i + \)%k and the sum of all grades is equivalent to the sum of 
the outputs modulo n. 


This protocol is implemented as a probabilistic program in both ra and m Here 
we implement it in standard ANSI C with the built-in non-deterministic functions of 
CBMC. The source code, shown on Figure 3.12, is based on the one provided in [ 27] . 
The array h[S] stores the grades of all students, i.e. the secret. The attacker can 
observe sum °/„ n. 


To compare QUAIL with our tool, sqifc, we repeat the experiment of the authors 
for the grade protocol with the tool and examples provided in [8]. However, QUAIL 
timed out after 1 hours for most of the cases, as showed in Figure 3.14 (we had the 
same results of leakage with the authors in the cases the tool did not time out). 
Therefore, we take the result in Figure 3.13 directly from the paper [27]. Comparing 
the results in Figure 3.13 it is easy to realise that the bounds on the leaks, measured 
by sqifc, do not exceed the real leaks, measured by QUAIL, by more than 1 bit, 


while sqifc required no more than 1 minutes in all cases as showed in Figure 3.14 
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1 

int func ( ) { 

2 

size_t S = 5, G = 5, i = 0, j = 0; 

3 

size_t n = ((G-1)*S)+1, sum = 0; 

4 

size_t numbers[S], announcements[S], h[S]; 

5 


6 

for (i = 0; i < S; i++) h[i] = nondet.int () °/« G; 

7 


8 

for (i = 0; i < S; i++) 

9 

numbers [i] = nondet.int () °/ 0 n; 

10 


11 

while (i<S) { 

12 

j=0; 

13 

while (j <G) { 

14 

if (h[i]==j ) 

15 

announcements[i] = 

16 

( j + numbers [ i] - numbers [ ( i +1) 7, S ] ) 7 0 n ; 

17 

j=j+i; 

18 

1 

19 

i=i+l; 

20 

} 

21 


22 

for (i = 0; i < S; i++) 

23 

sum += announcements [i] ; 

24 


25 

return sum °/ 0 n; 

26 

> 


Figure 3.12: The Grade protocol 


Tool 

QUAIL 

sqifc 

Students 

2 

3 

4 

5 

2 

3 

4 

5 

Grades 

2 

1.500 

1.811 

2.030 

2.198 

1.585 

2.000 

2.322 

2.585 

3 

2.197 

2.525 

2.745 

2.910 

2.322 

2.807 

3.170 

3.459 

4 

2.655 

2.984 

3.201 

3.365 

2.807 

3.322 

3.700 

4.000 

5 

2.999 

3.325 

3.541 

timed out 

3.170 

3.700 

4.087 

4.392 


Figure 3.13: Leakage measured by QUAIL and sqifc 


Tool 

QUAIL 

sqifc 

Students 

2 

3 

4 

5 

2 

3 

4 

5 

Grades 

2 

1.306 

241.483 

- 

- 

5.657 

7.029 

10.767 

9.469 

3 

28.613 

- 

- 

- 

9.145 

11.597 

17.987 

20.930 

4 

508.313 

- 

- 

- 

10.095 

16.872 

21.869 

18.579 

5 

- 

- 

- 

- 

14.639 

20.666 

33.298 

40.399 


Figure 3.14: Elapsed time in seconds of QUAIL and sqifc 
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3.6.5 The Dining cryptos protocol 


This case study is a variation of the dining cryptographers protocol of Chaum [36], 
one of the most popular problem in anonymity protocol. There is a group of cryp¬ 
tographers gathering around a table for dinner. After the meal, they are informed 
that the bill has been paid by someone, who could be one of them or the National 
Security Agency (NSA). Even though the cryptographers respect each other’s right 
to make an anonymous payment, they want to find out whether the NSA paid. To 
determine this, they use a protocol as follows: each pair of adjacent cryptographers 
toss a coin hidden from everybody else, so that each cryptographer only knows the 
values of the coin shared with the one on his left and with the one on his right; then 
each cryptographer declares aloud the exclusive OR of the two coins he sees, i.e. 0 if 
they have the same value and 1 otherwise. However if the payer is one of the cryp¬ 
tographers, he declares the opposite. In the end, if the sum of all declared values is 
even, then it is concluded that the NSA paid the bill. On the other hand, the sum is 
odd means one of the cryptographers did it. 
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size.t func(){ 

size_t N = 5, output =0, i = 0; 
size.t coin [N] , obscoin [2] , decl[N]; 
size.t h; 

h = nondet.uchar () 7, (N + l); 

for (i = 0; i < N; i++){ 

coinfi] = nondet .uchar () 7. 2; 

> 

for (i = 0; i < N; i++){ 

d e c 1 [ i ] = coin [i] ~ coin[(i + l)7,N]; 

if (h==i+l){ 

decl [i] = ! decl [i] ; 

> 

i = i +1; 

> 

for (i = 0; i < N; i++){ 

output = output + declfc]; 

> 

return output; 


Figure 3.15: The Dining cryptos protocol 
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We are interested in knowing how much information about the payer can be leaked by 
the sum of all declared values (in the dining cryptographers the observation are the 
declared values instead). The input code for the protocol is depicted in Figure 3.15 h 


is the identity of the payer, i.e. the secret, output is the observable. The coin toss is 
modelled by a built-in non-deterministic function in line 10. This model is less precise 
than implementation in probabilistic programs where it is possible to select random 
values from a specific distribution. By modelling with non-deterministic function 
and computing channel capacity, we can only compute the maximum leakage in all 


possible distributions. Figure |3.16| shows the channel capacity computed by sqifc, 
and the time to compute them. 


Cryptos 

3 

4 

5 

6 

100 

300 

Channel capacity 

2 

2.32 

2.59 

2.81 

6.658 

8.234 

Time in seconds 

2.145 

3.496 

3.632 

18.634 

158.517 

3326.915 


Figure 3.16: The dining cryptos protocol analysed by sqifc 


3.7 Discussion of related work 

Meng and Smith introduce an approximate technique to calculate an upper bound 
on channel capacity in [85]. The authors’ implementation of the method is largely 
manual, and we proposed an automation for it in p5]. While the work of Meng and 
Smith is very inspiring, the technique can be very imprecise, for example when the 
leaks are sparse in the state space. Moreover, the user policy is not encoded in the 
analysis which makes it infeasible when the leaks are not small. Take an example of 
a program that leaks all 32 bits of integral confidential data, it needs to make 64 calls 
to STP solver to determine that all bits are Non-fixed. Then, in order to determine 
two bit patterns of (31*32)/2 = 496 pairs of Non-fixed bits, it needs to make another 
496 * 4 = 1984 calls to STP solver, so it is 2048 calls in total. 

The first automated method for QIF was proposed by Backes et al. [11]. The method 
can be divided into two stages: first, it employs model checking to compute an equiva¬ 
lence relation 1Z on the set of confidential inputs w.r.t. observable outputs; secondly, if 
this relation 1Z can be represented by a system of linear integer inequalities Ax ^ b, 
which means it is a bounded integer polytopes , then a variant of Barvinok’s algo¬ 
rithm [23] can be used to count the number of integer solutions of 1Z. While this 
work is important as the first effort on automation of QIF analysis, it is not clear 
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however how this approach can be applied to real-world programs because of, for 
example, bit-wise operators in the CRC case study or non-linear relations and so on. 

Closer to our work is the paper of selfcomp j6j3j discussed in the previous section. 
However, as already outlined their approach to address the question "Does it leak 
more than k?" is quite different from ours. Kopf et al. m also apply QIF to real- 
world applications, i.e. leakage of cache side-channels; their technique is based on 
abstract interpretation and hence not based on bounded models. Because of this 
however they over-approximate channel capacity. 

A preliminary version of this chapter has been presented first in a workshop [98] . 
and then in a conference [95]. However our definition for ^SMT was a little bit 
different from the one in this chapter: we required that each of the Boolean variables 
in the set Vj is a Boolean abstraction of some T-atorn, hence we named the problem 
Propositional Abstract Model Counting. This requirement makes the definition more 
complicated and less general. 

A recent paper m explores QIF in a pure logical framework. The approach is 
powerful and elegant, however it is more limited when compared to our approach as 
it relies on the solver to generate models whereas our approach can use any solver 
instead. For example we can analyse Java by using JPF as a solver for bytecode even 
if JPF doesn’t generate a model in the sense of ra- 

McCamant and Ernst released FlowCheck [83], a tool for security testing based on 
dynamic taint analysis. What FlowCheck measures is the number of tainted bits, 
not an information-theoretic bound, so it is significantly different from our approach. 
Another tool is described in [89], it is able to analyse large programs using the notion 
of channel capacity in the context of dynamic taint analysis, while our approach 
is based on verification techniques. In this sense, our work comes with stronger 
theoretical guarantees. 
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Chapter 4 

Symbolic Execution as DPLL 
Modulo Theories 


The previous chapter has introduced the Symbolic Quantitative Information Flow 
(SQIF) approach, which mimics the DPLL(T) algorithm. SQIF was implemented on 
top of the Bounded Model Checker CBMC, used as a sub-routine, and can analyse 
programs written in C/C++. This chapter presents an alternative implementation 
for the SQIF approach, using Symbolic Execution. 

The implementation is based on a key observation that Symbolic Execution can be 
viewed as a variant of the DPLL(T) algorithm, or in other words, Symbolic Executors 
are SMT solvers. 

This view enables us to modify Symbolic PathFinder mm. a Symbolic Executor for 
Java bytecode, into a QIF analysis tool, jpf-qif, with little effort. The work in this 
chapter is the first to use Symbolic Execution for QIF analysis, and jpf-qif is the first 
QIF analysis tool for Java. 

4.1 Introduction 

Symbolic Execution (SE) [7TJ is now popular. It is increasingly used not only in 
academic settings but also in industry, such as in Microsoft, NASA, IBM and Fujitsu 
[33]. In the success of SE, the efficiency of SMT solvers pTJ is a key factor. In fact, 
while SE was introduced more than three decades ago, it had not been made practical 
until research in SMT made significant advances [33]. 


54 



Recall that most state-of-the-art SMT solvers, e.g. [53, 30], implement the DPLL(T) 
algorithm [90] which is an integration of two components as the following. The first 
component is a DPLL-based SAT solver, to search on the Boolean skeleton of the 
formula. The second component is a T-solver to check the consistency w.r.t. the 
theory T of conjunctions of literals. The path conditions generated by a Symbolic 
Executor, e.g. Symbolic PathFinder (SPF) [ 100] , are also conjunctions of literals. 
Therefore, when an SMT solver checks such a path condition, only the T-solver 
works on it, and the SAT component is not usec0 

On the other hand, a classical Symbolic Executor m can also be divided into two 
components. The first component, called Boolean Executor hereafter, executes the 
instructions, and updates the path condition. The second component is a T-solver 
(since the SAT solver is not used) to validate the consistency of the path condition. 
This thesis shows that a Boolean Executor does the same work as DPLL algorithm. 
Thus, SE is a variant of DPLL(T). This view is important since it connects two 
communities and can give an insight for future research. 


4.2 Illustration of DPLL(T) 

A complete formal description of the DPLL(T) algorithm can be found in, e.g., [ 90 ]. 
Here we briefly recall some background via a running example as follows. 

W := (“>(^0 > 5) V T) A ((x 0 > 5) V T 2 ) A (~>(x 0 > 5) V (xi = x 0 + 1)) A (4.1) 

(->(xi < 3) V T 3 ) A (->(xi < 3) V (x 2 — x 1 — 1)) A 
((ah < 3) V T 4 ) A ((xi < 3) V (y 1 = x 1 + 1)) 

p is a Linear Arithmetic formula. Boolean variables, T ... T 4 , are called Boolean 
atoms, and atomic formulas, e.g. (x 0 > 5), are called theory atoms or T-atoms. Any 
first-order formula p can be abstracted into a Boolean skeleton by replacing each 
T-atom in p with its Boolean abstraction. For the example above, we define new 
Boolean variables G\, G 2 , A 1: A 2 , A 3 for the Boolean abstraction of T-atoms, and the 
abstraction can be expressed as: 

BA := Gi = (x 0 > 5) A G 2 = (x! < 3) A (4.2) 

Ai = (xi = x 0 + 1) A A 2 = (x 2 — X! — 1) A A 3 — (y i = X! + 1) 

1 This claim is not for the decision procedure STP [62], which converts Bit Vector formulas into 
propositional formulas and solves them with a SAT solver. 
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As the result, we obtain a formula f p ( p stands for propositional) as the Boolean 
skeleton of f. Obviously, f is logically equivalent to f p A BA. 


f p := (- 1 G 1 V Ti) A (Gi V T 2 ) A (-.Gi V Ai) A (4.3) 

(~'G 2 V T 3 ) A (~ 1 G 2 V A 2 ) A 
(G 2 V T 4 ) A (G 2 V A 3 ) 


function DPLL(BooleanFormula <^){ 
fi = TRUE; status = propagate(<^,/i); 
if (status == Sat) return Sat; 
else if (status == UnSat) return UnSat; 
while (True) { 
l = decide^); 
f = f A l; 

status = propagate (</?, /z); 
if (status == Sat) return Sat; 
else if (status == UnSat) 
if (allStatesAreExplored()) 
return UnSat; 
else backtrack(<£>, /z); 


Figure 4.1: DPLL algorithm 

The DPLL(T) algorithm is the integration of the DPLL algorithm with a T-solver. 
The DPLL algorithm searches on f p , returning a conjunction of Boolean literal /z p . 
Replacing all the new Boolean atoms, Gi and Aj, in /i p with their corresponding 
T-atoms, we obtain the conjunction /i in T. The T-solver then checks whether /j is 
consistent with the theory T. Below is the illustration of DPLL(T) on f (for the 
limit of space, only decision literals are shown in /i p ): 


0. /i p = True 

1. ii p = G\ 

2. / = Gi A G 2 

3. ^ p = Gi 

4. /i 1 = G\ A — 1 G 2 


f P — (~<G 2 V T3) A (~ 'G2 V A2) A ( G 2 V T4) A (G2 V A3) 
<^ p = True ; T-solver(/z) = Inconsistent 
<^ p = ( — 1G2 V X3) A (~'G2 V A2) A (G2 V T4) A (G2 V A3) 
f P = True ; T-solver(/x) = Consistent 


The DPLL algorithm tries to build a model using three main operations: decide, 
propagate, and backtrack [54j. The operation decide heuristically chooses a literal 
l (which is an unassigned Boolean atom or its negation) for branching. The operation 
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propagate then removes all the clauses containing /, and deletes all occurrences of ->l 
in the formula; this procedure is also called Boolean Constraint Propagation (BCP). 
If after deleting a literal from a clause, the clause only has only one literal left ( unit 
clause ), BCP assigns this literal to True. If deleting a literal from a clause results 
in an empty clause, this is called a conflict. In this case, the DPLL procedure must 
backtrack and try a different branch value. 

At step 1 , G i is decided to be the branching literal, and the T-solver validates that 
( x 0 > 5) is consistent. BCP removes the clause (Gi V T 2 ), and deletes all occurrences 
of ->Gh. This results in two unit clauses T\ and Ai, so they are assigned to True, which 
means p p = GiATiAAi. Similarly, at step 2 G 2 is chosen, i.e. /i p = G\ A7\ AAi AG 2 . 
The T-solver checks the conjunction: // = (r 0 > 5) A T A ( X\ < 3) A ( X\ — x 0 + 1). 
This is obviously inconsistent, thus DPLL(T) backtracks and tries -iG 2 , which leads 
to a consistent model. 

Note that DPLL(T) refers to various procedures integrating DPLL and a T-solver. 
There are DPLL(T) procedures with integration schemas different from what we have 
described here. The interested reader is pointed to [ 105 ] for further references. 

4.3 Symbolic Execution as DPLL(T) 

Intuitively, a program can be encoded into a (first-order) formula whose models cor¬ 
respond to program traces. Symbolic Executors explore all program traces w.r.t. the 
set of program conditions, therefore they can be viewed as SMT solvers that return 
all (partial) models w.r.t. a set of Boolean atoms. 

In this thesis we only consider bounded programs, since this is the class of programs 
that SE can analyse. This means every loop can be unwound into a sequence if 
statements. In order to encode a program into a formula, all program variables 
are renamed in the manner of Static Single Assignment form [49]: each variable is 
assigned exactly once, and it is renamed into a new variable when being reassigned. 
In this way, assignments such as x = x + 1 will not be encoded into an unsatisfiable 
atomic formula. Under these settings, a program P can be modelled by a Symbolic 
Transition System (STS) as follows: 

P = (S,I,G, A, T) 
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S is the set of program states, / C S' is the set of initial states; each state in the STS 
models the computer memory at a program point. G is the set of guards and A is 
the set of actions; guards and actions are first-order formulas. 


An action models the effect of an instruction on the computer memory. Actions 
that do not update the computer memory (e.g. conditional jumps) are Boolean 
atoms, the others are T-atoms. T C S x G x A x S is the transition function, 
tij = (Si , gij, ctij, Sj) G T models a transition from state s* to state Sj by taking action 
dij under the guard g^. After a transition t i} : Sj —)■ Sj, the state Sj is exactly as the 
state Si apart from the variable updated by the action a l3 . 


Note that this STS models the program in more detail than the transition system in 


section 2.3 that will be used in other chapters. 


One way to encode a transition into a first-order formula is to present it in the form: 

= g^ —> a^, or equally = ->g^ V a t j. This encoding expresses that satisfying 
the guard implies that the action a l3 is performed. In this way, a program trace 
is defined as a sequence of transitions: 


toi A ti 2 A • • • A t(fc- i)k — (“’fl'oi V aoi) A (~^g \2 V CZ 12 ) ■ ■ • A (~^g(k-i)k V «(fc-i)fc) 

The semantics of the program is then defined as the set of all possible traces, or 
equally the set of all possible transitions, which can be represented as the following 
formula: 


<p= f\ Uj = f\ {-'gij v d .^) 

tijdzT tij£T 


(4.4) 


void test(int x, int y){ 
if (x > 5) { 
x + + ; 

if (x < 3) 
x-- ; 
else 

y = x + 1; 

> 

> 



Figure 4.2: A simple program and its associated STS. “if {x > 5)” is modelled by two 
transitions (so, (^0 > 5),Ti,si) and (so, _l (^o > 5 ),T 2 ,S 2 ); then “x++ v is modelled by 
(s 1 , {xq > 5),Xi = Xq + 1 , S 3 ); similarly for the rest of the program. 








Figure 4.2 depicts a simple example program and its associated STS. Encoding this 


STS following (4.4) results in the formula (4.1) that we have illustrated with DPLL(T) 
in the previous section. We now illustrate this example with SE. 


Similar to an SMT solver, a Symbolic Executor at a high level can be viewed as the 
integration of two components: a Boolean Executor (BE) to execute the instructions 
and a T-solver to check the feasibility of path conditions. For example, SPF has a 
parameter symbolic .dp to customize which decision procedure to use. If we set this 
parameter with the option no_solver then SPF solely works on the BE. 


function EXECUTOR(Program P){ 

PathCondition pc = TRUE; 

InstructionPointer i = NULL; 
update (P, i); 

if (i == Return) return; 

while (True) { 
l = decide(f); 
pc = pc A l ; 
update(P, i); 
if ( i == Return) 

if (allStatesAreExplored()) 
return; 

else backtrack(pc, f); 

}} _ 

Figure 4.3: A simplified Boolean Executor 

Figure |T3] depicts a simplified procedure of a BE. This procedure can be described as 
trying to build all path conditions using three main operations: decide, update and 
backtrack. The operation decide chooses a literal l, a condition (or its negation) of 
an if statement, for branching, adding it to the path condition. The operation update 
then symbolically executes a block of statement, i.e. no branching statement presents, 
updating the computer memory. When the BE reaches the end of a symbolic path, 
it backtracks to explore other paths. A Symbolic Executor, which is the integration 
of a BE and a T-solver, backtracks if the path condition is not satisfied. 

Both DPLL and BE rely on Depth-First Search, they are similar in the way they 
decide and backtrack]^} After choosing a literal, e.g. (xq > 5), BE executes the 
block it guards, i.e. T and X\ = Xq + 1. This is exactly the same as in DPLL: after 
choosing g l3 , for all the clauses V cijj), BCP deletes assigning a tJ to True. 

2 We consider DPLL in its simplest form, without non-chronological backtracking. 
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Therefore, the operation update does the same work as BCP, we can view a BE as 
implementing the DPLL algorithm, and SE as DPLL(T). 


4.4 SQIF by Symbolic Execution: 

With the view of SE as ^DPLL(T), we are able to make a Symbolic Executor work 
as SymbolicQIF with little effort. The key idea here is to enumerate all concrete 
values from symbolic executions. 



Figure 4.4: Partial exploration path of SQIF-SE for the data sanitization program 
from Figure 3.1 

For a program P that takes symbolic inputs ii,i 2 ,--i a , and produces an output O, 
the result of running SE on P is as follows: 



/l(*l,*2-4a) ^ PCi 

/ 2 (il,i 2 -,ia) if PC2 



where fi,f2,--fp are formulas over symbolic inputs ii,i2,--i a . pc\ , pc2 ,. .pep are the 
path conditions. Notice /j expresses a symbolic final value for O, i.e. in terms of SymEx 
instead of ?2--, i a ) we could write Uj(O) for a t e E. The following proposition 
was proved by King m 

Proposition 1 


Vi, j e [1, P\ A i ^ j, A pcj = _L 
which means that path conditions are mutually exclusive. 
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Definition 21 For a path condition pci obtained from SE, the concretization set of 
pci, denoted CSipcf), is the set of all concrete values of output O that can be reached 
by executing the program following pci. 


Consider again the illustrative example in Figure |3.1[ in which there are two path 
conditions: pc\ — H < 16 and pc -2 = H > 16. The corresponding concretization sets 
of these path conditions are: CS{pc\) = [8..23] and CS(pc 2 ) = [24.,2 32 ]. The set of all 
possible values of output O is formed by the union of concretization sets of all paths, 
and thus: 


N = 


U CS(pc 


1=1 


The set CS(pCi ) can be computed by inserting the code in Figure 3.4 at the end of 
the program and run SE: we add M conditions, each one tests whether bit 6j of the 
output O is 0 or 1. These M conditions test all the bits of the output O. Exploring all 
possible combinations of these conditions leads to enumerating all possible values of O. 
We denote by SQIF-SE the implementation of SQIF using SE. A partial exploration 


path of SQIF-SE is described as in Figure |4.4| SE as implemented by Symbolic 
PathFinder (SPF) returns a concrete values for each possible path. The number of 
distinct concrete values is the N that we need to count. 


SQIF-SE is implemented into a prototyping tool jpf-qif built on top of SPF. The 
tool works on Java programs. 


4.5 Soundness and Completeness 

SQIF-SE relies on a Symbolic Executor, and hence it is complete in programs with 
a bounded model of runtime behaviour, which means programs have no recursion or 
unbounded loops. These are well-known issues in SE and handling them is orthogonal 
to our work. SQIF-SE is also sound given a sound Symbolic Executor. 


4.6 Evaluation 

To evaluate jpf-qit we compare it against our previous implementation sqifc and the 
selfcomp technique. For case studies, we revisit the data sanitization program and 
the CRC programs in the previous chapter. Moreover, we consider the Tax program 
as follows. 
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4.6.1 Tax Record 


Balliu et al. [19] provided a very interesting case study of information flow security in 
Java programs derived from the EU-funded FP7-project HATS. The program contains 
8 classes/interfaces and 267 LoC. In our analysis we assume the year 2011-2012 basic 
tax rate in UK, which is applied for a person whose income does not exceed 35 
thousand pounds per annum, is F — 20 / 7 ( 2 ] Thus, the tax is less than 7 thousands 
pound per annual. We assume that donations to charities is below the amount of tax 
to be paid. Obviously, one cannot pay more than what one earns. Following [12] we 
are interested in leaks of a taxpayer’s income and donations to a Tax checker. 

QIF vs. Declassification 

Balliu et al. considered two cases of declassification: the first one, called taxCheckerl, 
is associated with the policy “ income x F% + donation > payment ”, and the second 
one, called taxChecker2, is associated with the policy “ income x F% + donation — 
payment ”. They claimed that: “The value declassified in the taxCheckerl case, resp. 
taxChecker2 case, is a lower bound, resp. upper bound, of the value revealed to the 
tax checker in the fixed tax rate variant. ” 

We notice in this sentence the use of terms like “value revealed” and “bound”: the 
authors were trying to describe quantitative concepts. From the result of jpf-qif, we 
can give hence quantitative answers to these questions. In the case of taxCheckerl , 
the observable is whether the payment is greater or smaller than the sum of the tax 
and donations, which means there are 2 possible outputs. This corresponds to the 
leak of 1 bit. In other words, the user policy or threshold k — 1. Regarding the 
taxChecker2 case, under the assumptions listed above, the leakage is upper bounded 
by 4.86 bits obtained in 24.988 seconds. 


Case Study 

LoC 

sqifc time 

jpf-qif time 

selfcomp time 

Data Sanitization 

< 10 

11.898 

20.695 

timed out 

CRC (8) 

< 30 

1.209 

8.386 

0.498 

CRC (32) 


8.657 

9.357 

timed out 

Tax Record 

267 

- 

24.988 

- 


Figure 4.5: Times in seconds, timeout is 30 minutes. means inapplicable. 


3 Since SPF can only handle conditions with integer values, we simplify the code by replacing 
(■income x 20)/100 with tax as an integer. The simplification we made does not change the secrecy, 
i.e. entropy, of income , so it will not affect the result of our analysis. 
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Results of the analysis are shown in Figure 4J3 In general, jpf-qif is slower than 
sqifc, although they are different implementation of the same algorithm. It is not 
surprising, as Java is always considered to be slower than C. Moreover, SPF is a 
virtual machine running on top of the Java Virtual Machine. Hence, there are more 
overheads in the implementation using SPF. 


4.7 Discussion of related work 

The correspondence between Symbolic Execution and the DPLL(T) algorithm was 
first briefly mentioned in our previous work |2H]- In that paper, we described a 
preliminary version of the DPLL-based algorithm in the previous chapter (Figure [333] ) . 
However, at that time the tool sqifc is not yet available. Instead, we presented a quick 
implementation of the algorithm with SPF. 

A year later, Brain et al. [2H] published an excellent paper showing the correspondence 
between DPLL(T) and Abstract Interpretation. Although it is believed that Symbolic 
Execution is a case of Abstract Interpretation we are not aware of any paper to 
discuss rigorously this relation. 


J h.ttp://en.Wikipedia.org/wiki/Symbolic_execution 
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Chapter 5 

Concurrent Bounded Model 
Checking 


In the chapters [3] and [4| we have introduced two implementations of the Symbolic 
Quantitative Information Flow approach: the first one is sqifc, which employs the 
Bounded Model Checker CBMC; and the second one is jpf-qif, built on top of the 
Symbolic PathFinder symbolic execution platform. A natural research question would 
be whether there is a relation between the two symbolic techniques: Bounded Model 
Checking and Symbolic Execution. 

This chapter studies this relation and introduces a methodology, based on Symbolic 
Execution, for Concurrent Bounded Model Checking. In our approach, we translate 
a program into a formula in a disjunctive form, and this design enables concurrent 
verification: a main thread running a symbolic executor, without constraint solving, 
to build sub-formulas, and a set of worker threads running a decision procedure for 
satisfiability checks. 

We have implemented this methodology in a tool called JCBMC, the first bounded 
model checker for Java. JCBMC is built as an extension of Java PathFinder, an open- 
source verification platform developed by NASA. JCBMC uses Symbolic PathFinder 
(SPF) for the symbolic execution, Z3 as the solver and implements concurrency with 
multi-threading. 

For evaluation, we compare JCBMC against SPF and CBMC. The results of the 
experiments show that we can achieve significant advantages of performance over 
these two state-of-the-art tools. 
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5.1 Introduction 


Model checking techniques are often classified in two categories: explicit-state or 
symbolic, depending on how they process the states of the system. While explicit- 
state model checking enumerates all possible states of the system explicitly, possibly 
on-the-fly aunt symbolic model checking represents sets of states symbolically, and 
hence more efficiently, by using Binary Decision Diagrams [ 83 ] or Boolean formulae 
[ 26 ] , SAT or SMT-based Bounded Model Checking (BMC) [ 26 ] unwinds the transition 
relation of a program for a fixed number of steps k and checks whether a property 
violation can occur in k or fewer steps. This bounded verification is reduced to a 
satisfiability check performed by a SAT or SMT solver. BMC is widely used in the 
hardware industry. 

For software, the application of BMC for ANSI-C is embodied in CBMC [ 35 , 33 ], 
which has been successfully used for many practical applications. In CBMC a C pro¬ 
gram containing assertions is translated into a formula (in Static Single Assignment 
form) which is then fed to a SAT or SMT solver to check its satisfiability. A satisfying 
assignment indicates that an error was found. 

Bounded model checking has not been explored so far for many other languages, 
including Java. However, explicit-state model checking tools such as Java PathFinder 
(JPF) |ij have been successfully used for the verification of many Java applications. 
Furthermore, there has been an explosion of symbolic execution [ 73 ] tools that have 
been used successfully for test case generation and error detection in the context of 
many high level languages, for example [63, Ml 11061132] , In particular, relevant for the 
work reported here, Symbolic PathFinder (SPF) (lOO j is a symbolic execution tool 
built as an extension of JPF, that provides a symbolic analysis for Java programs 
involving multi-threading and complex data structures. 

In this thesis, we describe an alternative methodology for BMC which is based on 
“classical” symbolic execution (SE) in the sense of King [ 73 ], Note that the way 
CBMC transforms a program into Static Single Assignment form can also be viewed 
as executing the program symbolically. However, this encoding is different from the 
SE of King and we evaluate the two encodings as part of the work reported here. Our 
methodology is not language specific and only relies on a symbolic executor for that 
language and an SMT solver. 
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By using symbolic execution we obtain a translation of a program and assertions 
into a disjunctive formula encoding the path conditions for each bounded (complete) 
path explored in the code. This suggests a simple concurrent verification strategy that 
relies on the observations that any subset of disjuncts in a disjunction F = F l V- • - VF n 
can be separately checked for satisfiability and whenever a subset is found to be 
satisfiable the satisfiability task can be stopped. Hence the verification of disjunctive 
formulas is naturally parallelizable. 

We have implemented this methodology in a tool called JCBMC, which stands for 
Java Concurrent Bounded Model Checker. The tool is built on top of SPF, and uses 
it to generate the disjunctive formula from the code (constraint solving is turned off in 
SPF itself) and while generating the formula it sends sub-formulas to multiple worker 
threads for satisfiability checking. JCBMC handles programs with multi-threading 
and recursive input data structures and relics on a standard SMT solver, namely 
Z3 [T3J for solving the constraints. Other solvers can easily be incorporated. One can 
even use different solvers for solving different path constraints in parallel. 

Although JCBMC is only a prototype, and concurrency is implemented by multi¬ 
threading but not parallelized yet, its performance, compared with existing tools, i.e. 
Symbolic PathFinder and CBMC, is remarkable. We summarize our contributions as 
follows: 

• A methodology for concurrent bounded model checking that is based on “clas¬ 
sical” symbolic execution and it is naturally parallelizable. 

• The methodology is language independent and supports assume-guarantee rea¬ 
soning. 

• A tool JCBMC, a concurrent bounded model checker for Java. 

• Experiments to show effectiveness of the tool for verification of programs with 
multi-threading and data structures. 

• Comparisons with bounded model checking and “classical” symbolic execution, 
as embodied by CBMC and SPF respectively. 


5.2 Illustrative example 

We illustrate our approach using the simple example program in Figure [5TT[ We want 
to check if the assertion in line 9 is valid for all possible inputs. Note that an analysis 
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using an explicit state model checker such as JPF would not be feasible, as this would 
involve enumerating all the possible inputs to the program. 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


void test(int x, int y){ 
if (x > 5) { 


x + + ; 

if (x < 3) 
x--; 
else 

y = x; 

> 

assert (x < 10 ) ; 

> 

Figure 5.1: A simple example 
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Figure A2 illustrates a part of classical SE with constraints solving on the program 
following the path (1,2, 3,4, 6 , 7). This analysis could be performed using e.g. SPF. 
Instead of concrete values, SE takes the symbols x 0 and y 0 as inputs and executes 
them just like concrete values. It also keeps track of the path condition pc which 
consists of the conditions true along that path and the symbolic environment a which 
maps variables into expressions over the input symbols xo, yo- Typically, whenever 
the pc is updated, SE checks the satisfiability of pc using an off-the-shelf solver. 


Initially, pc is true, and a maps inputs to theirs symbols. When SE reaches line 2, 
it updates pc as (xo > 5) since this is the condition to reach line 3 where a becomes 
x i —y Xq T 1. In line 4, the condition (x < 3) is translated in a to c = (xo + 1 < 3). 

At this point SE calls an SMT solver, and detects that pc I- <c because (x 0 > 5) h 

—'(xo + 1 < 3), therefore it jumps to line 6 with pc unchanged. 

In our approach for BMC, SE plays the role of generating the formula which encodes 
the program behaviour and the property to be checked. The satisfiability of the 
resulting formula will be checked separately by an SMT solver. Therefore, we execute 
SE without invoking constraint solving whenever pc is updated, and we postpone 
checking the pc until the end of the execution path. In this way, we can save the 
execution time of calling the solver, but the trade-off is that infeasible paths are also 
included. However, this will not affect the soundness of the analysis, since constraint 
solving is performed later. When SE reaches line 9 following the path {1, 2, 3,4, 6 , 7}, 
we have: 

pc = (x 0 > 5) A —i(x 0 + 1 < 3) 
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Here we reach the property V to verify, which is (:r < 10). We denote by V\ a the 
evaluation of V in the symbolic environment a. At this point, a maps x to Xq + 1 , 
which leads to the following: 


V\ a — (xo + 1 < 10 ) 

The property V is violated in this path if we can find a model for 

pc A _| 'P| (T = (xq > 5) A “i(xo + 1 < 3) A -i(a;o + 1 < 10) 


Setting xq = 11 will provide such a model. 


approach for the code in Figure 5.1 


is: 


The whole formula generated by our 


((aio > 5) A -i(aio + 1 < 3) A -i(aio + 1 < 10)) V 
((xo > 5) A (xo + 1 < 3) A ~>(xq < 10)) V 
(-i(x 0 > 5) A -i(x 0 < 10)) 

In general, we use SE to explore all possible symbolic paths up to a certain length, 
and then encode the program together with the property to check into a formula of 
the form: 

M 

\f(pci A-tpy 

i= 0 

where N is the number of paths that may trigger the error. This form allows us to 
divide the formula into blocks of D disjunctions: 

D—l 2D—1 kD—1 M 

V (p c * A -^k) v \J {pa A -fPU) ■ ■ • v \J (pci A _| 'P| (ri ) V \J (pci A _i 'P| fTi ) 

i=0 i=D i=(k—l)D i=kD 

In this way, we can solve the formula concurrently using several threads, each one 
solving a single block. A model of a single block is also a model of the formula, 
therefore the procedure stops when any of the threads find out a model. In JCBMC, 
after the main thread generates a sub-formula and passes it to a worker thread, it 
moves on to generate the next sub-formula, while the worker thread solves the given 
sub-formula concurrently. 


5.3 Concurrent Bounded Model Checking 


Our method for concurrent bounded model checking is illustrated in Figure 5.3 The 
inputs to the method are: a program under test, a property to verify and three 




parameters - B is the search bound, N is the number of workers and D is the number 
of disjuncts to give to one worker. The goal is to check if the property holds in the 
program, np to exploration bound B. 


Program 


B, D, N 


Main thread 

A disjunction of D 
path conditions 


Stop 


I 


Solver 


Solver 


Solver 


Controller 


Symbolic Execution 
(constraint solving off) 


Worker thread 


Worker thread 


Worker thread 


Figure 5.3: Concurrent Bounded Model Checking architecture 

5.3.1 Bounded Model Checking by Symbolic Execution 

The program under test is analysed using “classical” bounded symbolic execution with 
constraint solving turned off. This means that whenever a path condition is updated, 
we do not check its satisfiability, but rather continue the exploration. As a result, 
symbolic execution may explore infeasible paths, which will be checked later using 
constraint solving. Our approach can be used for the bounded verification of safety 
properties, which we assume have been reduced to checking assertions embedded in 
the code. Furthermore, our method supports both assume and assert statements to 
enable assume-guarantee style verification. The assumed conditions are simply added 
to the path conditions during the symbolic execution. 

The result of SE is a disjunction of path conditions, encoding constraints on the 
inputs to follow those paths, up to the pre-specified search bound. From among these 
paths, only the ones that may lead to assert violations are selected for solving. This 
is achieved by the controller which collects sets of D violating path conditions and 
sends them for solving to parallel worker threads, using off-the-shelf solvers. The 
workers start solving as soon as they receive the disjunctive formulas, which may 
happen while the symbolic execution is still exploring the program. The verification 
terminates as soon as one of the threads finds a satisfying assignment, in which case an 
error is reported, or when all the disjunctions are found to be un-satisfiablc, in which 
case the assertion holds (no error) up to the given bound. Note that if the symbolic 
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execution discovers no potentially violating paths (i.e. the error is unreachable), then 
no solving will be performed. 

5.3.2 Comparing our approach with BMC and SE 

Compared with classical BMC we use an explicit enumeration of paths, while BMC 
uses an implicit enumeration of paths. Although at first glance the implicit encoding 
should be better our experiments, even with sequential JCBMC, show that this is not 
the case. Furthermore, the explicit enumeration is easily parallelizable, with simple 
and natural load balancing for different threads. Crucially our approach stops as soon 
as a path leading to an error is found to be satisfiable, while with classical BMC, all 
the program needs to be explored. 

Compared with classical symbolic execution: we solve only in the end. So obviously 
the price to be paid is the exploration of infeasible paths. On the other hand, again, 
it is naturally parallelizable and constraint solving, which is one of the bottlenecks 
in SE, can be done in parallel, even with different solvers, with little coordination, if 
any, needed. 

In the following we describe in more detail our method. We start with a description of 
a sequential approach, to clarify how we use symbolic execution to built a disjunctive 
formula of the path conditions. Solving this formula happens after the symbolic 
execution, sequentially. 


5.3.3 Sequential Verification 


Our approach of employing SE for BMC is based on a simple observation. Suppose 
Sk is an error state in the transition system P. To determine the reachability of Sk 
from the initial state s 0 , BMC builds a series of transitions s 0 —>■ Si s*,, 


resulting in the formula in (2.4) of Section 2.4.1 On the other hand, SE builds the 
path condition pc which is a first order formula also characterising reachability of Sk 
from So- Therefore, both condition (2.4) and pc characterize all the inputs that reach 
the error. 


Suppose we want to prove formally that the program P satisfies, within given bounds, 
a property V represented by a set of assertions. This means whenever the program 
reaches an assertion point, the assertion needs to be valid, which means: f\{pCi “A 
V\a,)- This can be verified by checking the satisfiability of VG° C ; A _| T’| cri ). 
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1: function SymExBMC(<t, pc, l, i ) 

2: if (i > B) 

3: return 

4: Extract statement s at l 

5: while (s is not an if-statement A l ^ EOF) 

6: if (isSAT = T) 

7: return 

8: if (s is ‘assume c’) 

9: pc-k— pcAc\ a 

10: else if (s is ‘assert c’) 

11: Processor A _| c| fT ) 

12: else 

13: Execute the assignment s, update a 

14: l next(/) 

15: Extract statement s at l 

16: if (/ = EOF) 

17: return 

18: Extract {c,lj,l±} from if-statement 

19: i A- i + 1 

20: pc\ pc A c |(7 

21: SymExBMC(cr, pci, /y, *) 

22: pc 2 pc A —ic|cr 

23: SymExBMC(cr, pc 2 , /j_, i) 

Figure 5.4: Formula generation for BMC 


function PROCESS( 7 ) 

r r v 7 

Figure 5.5: Process error 
paths 


Proof 1 First notice one formula is the negation of the other one, i.e. 


f\{pci V\ ai ) = f\ (~ 'pc* V V\ a .) = /\^{pci A^Py = -■ V(pci A-.P| ff .) 


/ience if \/(pCiA^V\ ai ) is satisfiable there exists an i such thatpCiA^V\ ai is satisfiable 
i. e. an initial state leading to the path pci and not satisfying the property V\ ai . On the 
other hand if \f(pci A^V\ ai ) if not satisfiable then no disjunct pci /\^V\ ai is satisfiable 
hence there exists no initial state leading to an execution path satisfying the assertion. 


The algorithm in Figure 5.4 shows how to build the formula F = \J(pCi A ~ t 'P\ a f) for 
the GC language described in the previous section. In essence, the algorithm runs 
classical SE with no constraint solving for checking pc satisfiability. A statement of 
GC is determined by its location l, and the function next(l) returns the location of 
the next statement. An if-statement consists of a condition c, the location lj if c is 
true, and the location lj_ if c is false. In the recursive procedure SymExBMC as well as 
the function Process, all parameters are passed by value. In SymExBMC, the checking 
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from line [6] to line [7] is only used in concurrent mode. In sequential mode, isSAT is 
set to false, and it is not changed in the whole procedure. T is declared as a global 
variable, initialised to false (denoted by _L). k is global constant, defining the bound 
of BMC. 


Similar to standard SE, at the beginning the symbolic environment a maps program 
inputs to symbols, the path condition pc is initialised to true (denoted by T). The 
depth i of the recursion is initialised to 0. The procedure SymExBMC starts by ensuring 
the search depth i not to reach the bound B (line [2] to line [3]). As from line [4] to line 


15, it symbolically executes a basic block, i.e. without branching statement, of the 


program. The basic block ends by an if-statement or when the location l reaches 
the end of the source file (l = EOF). Assumptions and assertions are evaluated by 
the current symbolic environment (line [9j 11). When there is an assignment, the 
symbolic environment updates the mapping for the variable in the left hand side 
by the evaluation of the right hand side. At the end of the block, if there is an 
if-statement at the next location, SymExBMC is recursively called for both the then 
path and the else path. The condition is added to the path conditions without any 
checking of path feasibility. 


5.3.4 Concurrent Verification 


The simple algorithm presented in the previous sections will essentially enumerate all 
the possible feasible and infeasible paths through a program (up to a given bound), 
collect the path conditions for each path into a formula in disjunctive form and then 
invoke a constraint solver to check the satisfiability, all at once. The disadvantage of 
the approach is that it needs to enumerate all the possible paths through the program, 
which can quickly become expensive, especially if multi-threading is also considered. 
We therefore propose a concurrent algorithm that parallelises SymExBMC by delegating 
the satisfiability check of the disjuncts in T to worker threads. The concurrency of the 
algorithm relies on two parameters: the number of concurrent workers available and 
the number of disjuncts sent to each worker: the optimal choice is architecture and 
SMT solver dependent. In our experiments we found 200 disjuncts to be a reasonable 
choice. 


The main function is in Figure 5J3 It initialises T, pc , a exactly the same as in 
sequential mode of SymExBMC. Here, isSAT is a boolean variable shared between the 
threads, and can be modified (set to true) by them, d is also a global variable of 
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T, isSAT ■<— _L; pc <— T; d,i <— 0 
l first statement 
SymExBMC(cr, pc, l, i) 
if (i > 0 A isSAT = _L) 

Execute Run(r, isSAT) in worker thread 
if {isSAT = _L) 

Return Verification successful 
else 

Return Verification failed 

Figure 5.6: Main thread 


function RuN(r, isSAT) 
Run SMT-Solver on T 
if (r has a model) 
isSAT = T 

Figure 5.7: Worker thread 


function PROCESS( 7 ) 

r <— r v 7 

d i — d -\- 1 
if (d > D) 

Execute Run(r, isSAT) in worker thread 
r i — _L; d i — 0 

Figure 5.8: Process error paths for Concurrent BMC 


the main thread only to keep the number of current disjuncts. After initializing, the 
function SymExBMC is called. The main difference between sequential and concurrent 
mode is the function Process in Figure A8 and the checking from line [6] to line [7] 
in SymExBMC. In sequential mode, isSAT is always false, so SymExBMC keeps building 
the formula until it reaches EOF. In concurrent mode, when the number of disjuncts 
in T reaches a bound B , Process sends T for satisfiability check to a worker thread. 
Crucially this worker thread can run in parallel to any other running thread as they 
run completely independent tasks. 


Whenever a worker thread finds a model for its own V it sets the shared variable 
isSAT to true which will return control to the main thread and end the computation 
with Verification failed. If no thread sets isSAT to true then SymExBMC will 
eventually terminate and the remaining disjuncts (whose number is hence less than 
D) are sent for satisfiability check to a final thread. Verification successful is 
returned only if no thread has set isSAT to true during the computation. 
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Implementation 


Our prototype tool, JCBMC, has been implemented following the Observer Design 
Pattern SPF executes symbolically the Java bytecode program and acts as the 
subject. The Controller acts as the observer, it waits for SPF to have generated a 
sub-formula T with D disjuncts and then sends it to an available worker thread. D is a 
user chosen parameter of JCBMC. The worker thread executes Run(r, isSAT) which 
first writes T into a file in SMT2 format [23], then calls the SMT solver Z3 to check 
for satisfiability. JCBMC creates a thread pool of N workers; current architecture 
doesn’t support parallelism but only multi-threading. 

JCBMC is built on top of SPF and as an extension of the JPF platform, therefore it 
inherits all the power of JPF and SPF. 


5.4 Evaluation 


Our evaluation comprises cases studies to compare JCBMC with SPF (with default 
configuration) and case studies to compare JCBMC with CBMCQ To compare with 
CBMC we have considered C code whose Java translation is almost literal. 


An important reminder is that the current implementation of JCBMC is multi¬ 
threaded but not yet parallel and we expect a parallel implementation to have a 
significant advantage over the current one. 


By JSBMC we denote the sequential implementation of JCBMC where a single thread 
is used. Experiments are run on a machine equipped with dual Xeon(R) E5-2670 


CPUs. The results are shown in Tables 5J3 and 5.10 Unless otherwise specified 
times are in seconds, xmy means x minutes and y seconds, “timed out” is one hour 
and x denotes a memory hitO The source code for the examples can be found at: 
https://github.cora/qsphan/jpf-bmc. 


1 To compare both tools with the same solver in the experiments CBMC will be called with option 
-smt2, and we will use Z3 for satisfiability checks. 

2 A memory hit is a “run out of memory” problem. This can be addressed by a different memory 
manager in JPF or with a direct implementation of SymExBMC. 
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SPF 

JSBMC 

CBMC 

JCBMC (10) JCBMC (200) 

Array size 

Bubble sort with assertion negated 

5 

4.517 

2.174 

0.460 

1.097 

1.338 

6 

5.622 

12.604 

0.817 

1.160 

1.389 

15 

36.194 

X 

56m34.033 

1.195 

1.948 

30 

4m32.790 

X 

timed out 

1.387 

2.905 

100 

timed out 

X 

timed out 

4.944 

34.697 


Verification of bubble sort 

5 

6ml9.222 

3.712 

7.171 

4.193 

3.622 

6 

timed out 

26.293 

37.816 

29.512 

21.834 

7 

X 

X 

5m22.641 

X 

X 

8 

X 

X 

timed out 

X 

X 

Sum of array 

unsafe 

1.403 

12.671 

lm5.738 

1.576 

2.479 

safe 

failed 

12.030 

2.252 

9.466 

10.614 


Figure 5.9: Performance of all tools. JCBMC(IO) and JCBMC(200) mean JCBMC 
is run with the parameter D as 10 and 200 respectively, “failed” refers to SPF failing 
to solve the constraints using the integrated solver. 

5.4.1 Comparing with CBMC and SPF 
Bubble Sort 


We consider the classical bubble sort algorithm, which has already been studied in the 
BMC community mm- Here, differently from na. we consider the more challenging 
symbolic version where the values of the array are non-deterministically chosen. We 
consider both the verification of the assertion “the elements of the array are ordered 
after bubble sort” and its negation “the elements of the array are not ordered after 
bubble sort”. We analyse a program implementing bubble sort. It will hence contain 
no bugs for the positive assertion and will be buggy for the negation. Results are 


shown in Fig £79 We notice that while CBMC is better for the positive assertion, 
JCBMC outperforms the other tools for the negative assertion and is capable of find 
a counterexample for array sizes of a higher order of magnitude. 


Sum of array 

We consider the array case studies taken from [2], in particular sum_ array_safe.c 
for verification and sum_ array_unsafe.c for refutation. The array size is set to 1000. 
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Results show both SPF and JCBMC outperform CBMC for the unsafe version, while 
CBMC has a slight advantage for the safe version. 

5.4.2 Comparing with SPF (Java code) 

The following examples consist of substantial Java code which is not naturally trans¬ 
latable in C; we hence compare JCBMC only with SPF. Notice JCBMC and SPF 
are both extensions of JPF: in the case all inputs are concrete they both reduce to 
JPF-core hence their performance is identical. Hence we only consider programs with 
symbolic inputs. 

Flap controller 

This case study is shipped with the distribution of SPF. It is a multi-threaded program 
modelling a simplified flap controller on an aircraft. It contains 3 classes, and 80 lines 
of code. This example demonstrates handling of multi-threading. 

Red Black Tree 

This is another example from the SPF distributions (3474 LOG in one class). We 
check for consistency of the tree after performing put, remove, get and firstKey 
symbolically. Results show that both JSBMC and JCBMC significantly outperform 
SPF. 

MER Arbiter 

The MER Arbiter models a component of the flight software for NASA JPL’s Mars 
Exploration Rovers (MER). The MER Arbiter has been modelled in Simulink/State- 
flow and it was automatically translated into Java using the Polyglot framework and 
analyzed with SPF ra- The configuration for our analysis involved two users and five 
resources. The example has 268 classes, 553 methods, 4697 lines of code (including 
the Java Polyglot execution framework) but only approx. 50 classes are relevant. We 
analyse the code with and without the error (see mi)' The performances of SPF, 
JSBMC and JCBMC are comparable, with SPF slightly better. 
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Tool 

SPF 

JSBMC 

JCBMC (10) 

JCBMC (200) 

Flap controller (unsafe) 

1.141 

2.899 

0.948 

1.370 

Red-black tree (safe) 

53.602 

3.942 

3.267 

2.774 

MER Arbiter (unsafe) 

5.275 

8.111 

7.479 

7.579 

MER Arbiter (safe) 

47.065 

59.145 

57.740 

58.886 


Figure 5.10: Performance on Flap controller, Red-black tree and MER Arbiter. 
JCBMC(IO) and JCBMC(200) mean JCBMC is run with the parameter D as 10 
and 200 respectively. 


Discussion of experiments 

Comparing with CBMC, JCBMC scores better in finding counterexamples than in 
verifying their absence; this is consistent with its design because a counterexample 
corresponds to a worker thread finding a model of the formula. Compared with 
SPF, JCBMC can be much better (see bubble sort or red black tree) but can also 
be comparable or slightly worse (see MER Arbiter and Flap Controller results). The 
reason for the latter is that the cost of generating path conditions dominates the cost 
of solving them. Similarly, SPF failed to generate formulas for bubble sort for sizes 
7 and higher. Furthermore, an error path (e.g. in MER) may occur at the beginning 
of the SE exploration, and it is therefore discovered quickly by SPF, while JCBMC 
still needs to generate the pre-specificd number D of error paths before solving them. 
The results suggest one direction for future work, namely to investigate improving 
the cost of SE-based path generation (see last section). 

5.5 Discussion of related work 

Related approaches on parallelising BMC [14, I115J address parallel solving of the 
conjunctive formula that is built for BMC and aim at performing solving at different 
bounds, where some clauses are shared to enable more efficient SAT solving. In 
contrast we aim to solve the formulas generated with SE for the same bound, which are 
naturally disjoint resulting in a simpler parallel algorithm. Furthermore our work aims 
at verifying programs written in high-level languages such as Java and it is not clear 
how the previous work, performed in the context of finite state automata, would be 
applicable. Also related is the work on parallel SAT and SMT solving [IU9, 11041UT6] . 
which can be seen complementing the work presented here, in the sense that we can 
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use e.g. the parallel version of Z3 [116] in each of the workers to further speed up our 
proposed approach. 

PKIND [ITT] is a parallel model checker for Lustre that uses k-induction. PKIND 
runs in parallel the different tasks involved in performing the induction, e.g. the 
base step, the induction step and also the generation of auxiliary invariants used 
for verification. Therefore PKIND performs the parallel work at a higher level of 
granularity than JCBMC. It would be interesting to investigate if we can replace 
the parallel tasks in PKIND with our own version of SE-based bounded verification, 
which in turn is parallelized at the level of granularity of symbolic paths. 

Parallel model checking has been investigated in the context of explicit-state [2m EU 
f3Ti. ;68. TT21 [70] and symbolic [38] [78] exploration. The latter were done in the context 
of verification using Binary Decision Diagrams, and hence are very different from 
ours. These approaches concentrate on partitioning the state space to be explored in 
parallel and on dealing with the communication overhead between parallel workers. 
In contrast, in our approach the workers perform the solving independently, with no 
communication between them. 

In previous work we have developed a framework for performing parallel symbolic 
execution in SPF mu. We used a set of pre-conditions to partition the symbolic 
execution tree to distribute its processing. These pre-conditions were computed a 
j priori , using a “shallow” symbolic execution up to a small exploration bound, to 
statically compute the different partitions of the input space with no communication 
overhead. Other approaches to parallel symbolic execution [731 221 1108] operate pri¬ 
marily by dynamically partitioning the symbolic execution tree for load balancing, 
which may result in better use of computational resources but also in more communi¬ 
cation overhead. All these approaches were done in the context of “classical" [1111173] 
or dynamic [321 ITUS] symbolic execution, using constraint solving during path gen¬ 
eration. In contrast the approach we advocate here has a clear separation between 
path generation and constraint solving, allowing us to easily achieve load balancing 
between workers, with little communication overhead. It would be interesting to com¬ 
pare experimentally and to also combine the techniques in JCBMC and the parallel 
version of SPF and we plan to do that in future work. 
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Chapter 6 

Quantifying Information Leaks 
using Reliability Analysis 


In chapter [4[ we have presented the first technique to use Symbolic Execution for 
Quantitative Information Flow analysis. There, Symbolic Execution has been used for 
both exploring the program and counting the models. In this chapter, we explore an 
alternative approach which uses Symbolic Execution only for exploring the program, 
leaving the task of counting models for a reliability analysis tool. 

This chapter can be divided in two parts. The first part studies the qualitative aspect 
of information flow analysis. Its contribution is a novel and practical approach for 
self-composition using Symbolic Execution. 

In the second part of the chapter, we show the relation between reliability analysis 
and Quantitative Information Flow. Exploiting this relation, we combine our new 
self-composition technique with a Symbolic Execution-based reliability analysis tool 
to quantify information leaks in Java bytecode. 


6.1 Introduction 


Recall that in section 2.1.1 we have introduced the theorem proving approach to non¬ 
interference, in which non-interference is logically formulated by self-composition. 
As far as we are aware, this has been the only approach for qualitative analysis 
that returns neither false positives nor false negatives. We also quoted the claim of 
Terauchi and Aiken [ 113] that self-composition was impractical and that it would 
be unlikely to expect any future advance. The main limitations of self-composition, 
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as pointed out by Terauchi and Aiken, come from the symmetry and redundancy of 
the self-composed program, which lead to some partial-correctness conditions that 
hold between P and Pi. To find these conditions is crucial for the effectiveness of the 
analysis, however, finding them is in general impractical. He we present a practical 
implementation for self-composition. 

6.1.1 Qualitative analysis 

The idea of self-composition is to have a copy Pi of the program P to compare with 
itself. The approach can be divided into two steps: the first step is to compose 
the program with a copy of itself; the second one is to perform analysis on the self- 
composed program. Our approach is to delay self-composing to the second step: first, 
we perform analysis on the original program with Symbolic Execution; second, we 
self-compose the result of the analysis to get the formula of self-composition. 

We expand the idea of comparing the program P with it copy Pi into comparing 
all pairs of executions p of P and p\ of Pi. Since it is impossible to enumerate all 
possible executions, we use Symbolic Execution to synthesize the symbolic paths that 
represents a set of concrete executions, and perform comparison on these symbolic 
paths, which we formulate as path-equivalence. 

The delay of self-composing after performing the analysis is the main novelty of 
our approach. In this way, we could avoid the symmetry and redundancy of the self- 
composed program. Moreover, the symbolic paths synthesized by Symbolic Execution 
are presented by first-order theories, just as the generated formula of self-composition. 
The validity of this formula can be automatically and efficiently checked by powerful 
SMT solvers. 

6.1.2 Quantitative analysis 

Traditional self-composition technique can only tell if a program leaks information. 
On the contrary, we can refine our Symbolic Execution-based self-composition into a 
more fine-grained analysis that can decide if a path of the program leaks information. 
We then quantify the leaks for each symbolic path using a reliability analysis tool. 

Our approach is implemented into an automated tool, called QILURA (which stands 
for Quantify Information Leaks Using Reliability Analysis). Given a program, and 
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inputs labeled as high and low, QILURA computes an upper bound on the maximum 
number of bits that the program can leak to a public observer. Our implementation is 
done in the context of Java bytecode programs and the SPF [ 100] symbolic execution 
engine, extended for reliability analysis [32]. However, the work is general and can be 
applied in the context of any programming language for which a symbolic execution 
tool exists. 



Figure 6.1: Architecture of QILURA 


At a high level, the architecture of QILURA is depicted in Figure 6.1 The user labels 
the inputs of the program with high and low. The program is then passed to SPF 
to collect all possible symbolic paths. The Labeling Procedure , using a fine-grained 
self-composition [22], classifies all the paths into three categories: clean, direct and 
indirect. The procedure uses z3 [321] for satisfiability checking of self-composition 
condition. 


Finally, the Quantifying Procedure uses Barvinok model counting techniques [5] over 
the symbolic constraints (simplified using Omega [TJ ) collected by SPF to count the 
number of inputs that follow paths labeled with “direct” and provides an upper bound 
of k bit on the leaks. 


6.2 Preliminaries 

This section reformulates Symbolic Execution and provides some background on the 
new Symbolic Execution-based reliability analysis framework of Filieri et al. [59]. 

6.2.1 Symbolic Execution 

In chapter [4] we described Symbolic Execution on a Symbolic Transition System 
which modelled in detail the transition of a program with guards and actions. These 
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details are not necessary in this chapter and are not captured in the transition system 


in section [273| that we use to describe concrete execution. Therefore, we reformulate 
Symbolic Execution in a more coarse-grained transition system that only takes in to 
account the source and target states of a transition. A program P is modelled as 
follows: 

P = (S*, /*, F*,T*) 


where S* is the set of symbolic states; each s* G S* represents a set of concrete states 
s G S. I* C S* is the set of initial symbolic states; F* C S* is the set of final symbolic 
states; and T* C S* x S* is the transition function. A symbolic path (symbolic trace) 
of the program P is represented by a sequence of symbolic states: 


such that Sq G I*, s* k G F* and (s*, s* +1 ) G T* for all i G {0,..., k — 1}. The symbolic 
semantics of P is then defined as the set of all symbolic paths 7 Z*, which is also called 
as the symbolic execution tree. Likewise, each p* G 1Z* represents a set of traces 
pell. 


We denote by X\ y the value of the variable X at the state y. After symbolically 
executing the program P with initial input symbols H = a, L = /3, for each s* G F*, 
i.e. each leaf of the symbolic execution tree, we have a symbolic formula for the value 
of the output 0 in the symbolic environment: 

0\ s * = fi(<*,P) 

Another product of SE is the path condition pet = Ci(a,/3 ) for s* to be reachable. 
Each pci corresponds to a symbolic path p*. The following theorem was also proved 
by King m 


Theorem 3 


Vi, j G [1, n\ A i ^ j.pci A pcj = _L 


We define the function path such that: 

path(p*) = pet 

The output 0 can be considered as a result of the following function: 

fi{<*,P) if Ci(a,P) 

f 2 {oi,P) if c 2 {a,P) 

fn(a,P) if c n (a,P) 

Or the following always holds: 
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Corollary 2 

Vi e [1, n\.Ci(a, f3) ->0 = fi(a,/3 ) 

fi and c, are in general combination of first-order theories, e.g. linear arithmetic, 
bit vector and so on. SE tools make use of off-the-shelf SMT solvers to check the 
satisfiability of q, and eliminate unreachable paths (which may appear in the control 
flow graph). 

6.2.2 Reliability Analysis 

Reliability analysis jifJTj aims to compute the probability that a program successfully 
accomplishes its task without errors. Most previous work perform reliability analysis 
at early stages of design, on a architectural abstraction of the program, and thus they 
are not applicable to source code. 

In [53], Filieri et al. introduced the first approach that can compute the reliability 
of program from Java bytecode. Their approach is to use SE to enumerate each of 
the symbolic paths (and its path condition pq). The symbolic path is then labelled 
as: (i)T if the program accomplishes the task; (ii) F if the program reaches an error 
state; (iii) G if we cannot decide because the path is not fully explored (G stands for 
grey). 

From the path condition pq, Filieri et al. use the tool Latte [5] to compute efficiently 
the number of inputs #(pq) that satisfies the path condition pc. The reliability of 
the program, i.e. the probability that the program accomplishes its task, is then 
computed as: 

v = _ s #(pc r ) _ 

E#(pc r ) + E#(pc F ) + S#(pc G ) 

For QILUR.A we do not compute probabilities but use directly the counts over the 
computed symbolic constraints. 

6.3 Self-composition by Symbolic Execution 

To avoid the limitation of the theorem proving approach, we need to reformulate the 
self-composition formula into a simpler logic which does not contain the program P. 
This is made possible by using the trace semantics of programs. 
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6.3.1 Self-composition as path-equivalence 

Given a program P that takes secret input H, public input L and producing public 
output 0; Pi is the same program as P, with all variables renamed: H as Hi, L as Li 
and 0 as 0i. The trace semantics of P and Pi are 1Z and 1Z\ respectively. 

Definition 22 (trace-equivalence) The program P satisfies non-interference if: 

Vp £ 7Z, p\ £ 1Z\.L\ inipp'j Ti|j n jppp t 0\fin(p) Ol\fin(pi) (6.2) 


It is stated similarly to the Hoare triple in (2.1): for all possible pairs of traces p of P, 
and pi of Pi: if L = L\ at the initial states, then O = Oi at the final states. At this 
point, we have a formulation of self-composition that does not involve the programs 
P and Pi. 


However, even with simple programs, it is impossible to compute all the traces. Our 
solution is to use trace-equivalence with SE. Recall that each symbolic path represents 
a set of traces, and it is possible to build a complete symbolic execution tree (here 
we only consider bounded programs). Following Corollary |2j trace-equivalence in the 
context of SE is redefined as follows: 


Definition 23 (path-equivalence) The program P satisfies non-interference if and 
only if for all p* £ 1Z* and for all pi £ 7 Z\, the following equation holds: 


(-^ | imt(p*) |imt(pj)) A path(p ) A path(pf) y (G| /in(p*) t^l|/m(p*)) (6-3) 


In this way, we have an SMT formula, i.e. a combination of first-order theories. 
This is the key novelty of our approach, since the formulation of self-composition in 
first-order theories enables us to solve it efficiently using off-the-shelf SMT solvers. 


6.3.2 Path-equivalence generation 

Suppose P is symbolically executed with H = a, L = f3. To simplify the formula, we 
choose the input symbols for Pi as Hi — op, Li — /3 so that L\i n u( p *) = Tulmittp*) is 
automatically satisfied. That means: 

(H\i n it(p*) Ol) A (T|init(p*) ft') A (Hi | i n it(p*) Op ) A (-7u|mit(p*) A) 
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Given the result of SE is a function of the output 0 as in (6.1), the path-equivalence 
in (6.3) can be rewritten as: 


PE = DF A IF 

where: 

n 

DF = A Ci(a, (3) A Ci(ai, f3) —> (/,(«, /3) = /*(«!, /3)) 

Z=1 

n—1 n 

IF =A A G(a,/3) Acj(q:i,^) ->• (fi(a,P) =fj(a u p)) 

i= 1 J=i+1 

HF checks the path-equivalence when both P and Pi follow the same symbolic path, 
and thus it guarantees the absence of direct flows. On the other hand, IF checks the 
path-equivalence when P and Pi follow different symbolic paths, and it guarantees the 
absence of implicit flows. 

6.3.3 Examples 

We illustrate the approach with some toy examples. Here we assume the same setting 
as above: a program P with confidential input H, public input L, and output 0. SE 
executes P with input symbols H = a and L — j3. 

Implicit flow 

Consider the password checking program: By SE, we have: 

if (H == L) 

0 = true; 

else 

0 = false; 


DF and DF are 


O = 


true 

false 


generated as follows: 


if a = /3 1 
if a fz (3 J 


DF = (a = (3 A a\ = (3 —> true = true ) Afa^Aai^A false = false ) 
IF = a = f3Aai^f3— * true = false 


It is trivial to prove that DF is valid and IF is invalid, and thus the program violates 
non-interference and leaks information via implicit flows. 
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No flow 


Consider the modified version of the password checking procedure in Listing 2.1.1 


if (H == L) 

0 = false ; 

else 

0 = false; 


By SE, we have: 


O = 


false 

false 


DF and IF are generated as follows: 


if a = /3 1 
if a 7^ f3 j 


DF = (a = /3Aa\=/3 —» false = false ) A (a (3 A ol\ ^ j3 — > false = false ) 
IF = a = /3Aai^f3— > false = false 


It is trivial to prove that both DF and IF are valid, and thus the program satisfies 
non-interference. Note that this is the case that type systems, taint analysis would 
decide as violating non-interference. 


No confidential data involved. 

Consider the password checking program, with a small modification to exclude the 
confidential data in its computation, i.e. to make it secure. 

if (L == 3) 

0 = true; 

else 

0 = false; 


Similarly we have: 


O 


DF and IF are derived as: 


true if /3 = 3 

false if —i (/3 = 3) 


DF =(/3 = 3A/3 = 3—> true = true ) A ( _i (/3 = 3) A -i(/3 = 3) — > false = false) 
IF = f3 = 3 A -i ({3 = 3) —> true = false 


Both DF and IF are valid, which confirms the intuition that the program is secure. 





Both implicit and explicit flows 


Consider again the data sanitization program: 


if (H 

< 16) 

0 

ii 

EC 

+ 

r 1 

else 


0 

= L; 


The summaries and path conditions returned by SE are as follows: 


O = 


ol T f3 

P 


DF and DF are generated similarly: 


if a < 16 1 

if -i(a < 16) J 


DF = (a < 16 A cti < 16 —* a + f3 = a\ + (3) A (-i(a < 16) A ->(ai < 16) — > /3 = j3) 
IF = a < 16 A -i(«i < 16) —» a + /3 = 

It is easy to find counterexamples to make DF and IF invalid, for example: (a = 
1; = 2) for DF and (a = 1; ot\ = 17) for IF. So the program leaks via both implicit 

and explicit flows. 


6.3.4 Optimization 


After SE collects the symbolic paths and path conditions as in (6.1), there are three 
possible cases for a p* € IZ*\ 


0\fin( p *) and path(p*) does not contain a: p s can be classified as “secure”, and is 
excluded in computing DF and IF. With this optimization, programs like the 


one in Figure 2.1.1 can be classified as secure without computing anything. 


• 0\fin( P *) does not contain cc; path(p*) contains a: in this case, computing DF for 
this path will result a formula C{f3) —> true which is valid for any C. Therefore, 
p* is excluded in computing DF. 


• 0\fi n (p*) contains a: we can conclude that the program is insecure, leaking 
information via direct flow. This is very similar to taint analysis, however SE 
is more precise, since it can detect cases such as H x 0 or H — H and so on. 
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These three optimizations are sufficient to eliminate the computation of DF, and 
simplify the formula to be validated. 

In a previous paper [ 113] . Terauchi and Aiken presented an interesting program that 
computes Fibonacci numbers, and containing confidential data. Applying the self¬ 
composition technique for this program turned out to be very tricky because of the 
symmetry and redundancy of the self-composed program, and the state-of-the-art 
safety analysis tool BLAST failed to terminate. With our Symbolic Execution-based 
approach and these optimizations, the case study becomes trivial, and can be checked 
quickly without computation of DF and IF. 


6.4 Quantifying Information Leaks by combining 
Self-composition and Reliability Analysis 

At a high level, QILURA performs a two-step analysis. First, SE is run to collect 
all symbolic paths of the program (up to a user-specified depth), then each path 
is assigned a label: (i) clean: if it leaks no information, (ii) direct: if it leaks 
information via direct flow, and (iii) indirect: if it leaks information via indirect 
flow. Secondly, a model counter for symbolic paths from [59] is used to count the 
number of possible inputs that go to “direct” paths, and compute an upper bound on 
the leakage. QILURA is available at: https://github.com/qif/jpf-qilura. 


6.4.1 Fine-grained self-composition 


Checking the satisfiability of the path-equivalence condition PE = DF AIF in section 


6.3.2 can only decide whether a program leaks information. However, we can refine 


it to a path-level analysis of leakage. 


We assume the same settings as in section 6.3.2 we symbolically execute the program 
P with the input symbols: H = a, L = /3, and the program Pi with the input symbols 
Hi = aq, Lx = f3. Based on the path-equivalence condition for direct flow and indirect 
flow, we define the self-composition condition for each symbolic path as follows. 


Definition 24 Given a path p* such thatpath(p*) = Ci(a, (3) and 0\fi n ( p *) = fi(a,(3), 
p* does not leak information via direct flow if and only if the following condition holds: 


d(a, (3) A Ci(oti, (3 ) (fi(a, f3) = /i(aq, (3)) 







Applying the material implication rule on the condition above results in: 


-i (ci(a,P) A Ci(a i,/3)) V (/*(a,/3) = fi(a 1} (3)) 

In case p* leaks information via direct flow, the condition above is violated, which 
means its negation is satisfiable: 

-i(-i(ci(a, P) A Cj(o!i, P)) V (fi(a,P) = fi(a u P))) 

This formula can be simplified using De Morgan’s law as: 

Ci(a,P ) A Ci(ati,P) A ->{&(<*, P) = fi(ati,P)) (6.4) 


Definition 25 Given two symbolic paths p* and p* such that path(p*) = Ci(a,P), 
0\fin( P *) = fi{ot,P), path(p*j) = Cj{a,P) and 0\ fin ( p * ) = fj{a,P), p* and p* do not 
leak information via indirect flow if and only if the following condition holds: 

Ci(a,P ) A Cj(a!i, P) -)• ( fi(a,P ) = f^a^P)) 


Similar to the derivation above, in case p* and p* leaks information via indirect flow, 
the following formula is satisfiable. 


Ci(a, 0) A Cj(a u 0) A 0) = fj(a i, /9)) 


(6.5) 


Based on the conditions in (6.4) and (6.5), we implement a procedure to label all 
symbolic paths as in Figure 6.2 The function isSAT is implemented by calling the 
SMT solver z3 


The algorithm of this procedure is straightforward. At the beginning, all paths are 
labeled as being clean. The procedure then searches for direct flow by checking 
the condition (6.4) for all paths. It then searches for indirect flow by checking the 


condition (6.5) on all possible pairs of symbolic paths. 


6.4.2 Model Counting for Symbolic Paths 

For a symbolic path p, let ffin(p) and ffout(p) denote the number of concrete inputs 
and outputs of p respectively. Obviously ffin(pi) is #(pq) computed in 159] - After 
being labeled, all paths are classified into three categories: clean, direct and indirect. 
So the channel capacity is bounded by: 

CC(P ) < log 2 (S#out(p c ) + S#out(pi) + S#out(p d )) 


where p c is the clean path, p^ is the indirect path, and pd is the indirect path. 







for all pi do { 

label[i] A- clean 

p A- Ci(a , f3) A d(a 1, /?) A /3) = fi(a u /?)) 

if (isSAT(<^)) label[i] A- direct 

end for} 

for i = 1 to n — 1 do 

for j = i + 1 to n do { 

tp A- a(a, /3) A Cjipt!, p) A -i(/i(Q!, /3) = fj(a x , /3)) 

if (isSAT(<^)) { 

if (label[i] = clean) label[i] indirect 
if (labelfj] = clean) label [j] indirect 

end for} 
end for 

Figure 6.2: Fine-grained self-composition 

• Since clean paths are not interfered by the confidential input we can replace 
S#out(p c ) with 1. 

• An indirect path only reveals that the program follows that path, its output 
is not interfered, and each path has one output. Thus, S#out(pi) is just the 
number of indirect paths. 

• We hence only need to compute S#out(pd)- 

A deterministic program can be viewed as a function that maps each input to ex¬ 
actly one output (denotational semantics). Therefore, the number of inputs is always 
greater than or equal to the number of possible outputs. This means #in(p) > 
#out(p), and S#in(p d ) > S#out(p d ). 

By using the model counting engine for symbolic paths in [59], we can compute 
S#in(p d ), and hence compute an upper bound of channel capacity CC(P). 


6.5 Evaluation 

Automated QIF analysis is notoriously hard. To the best of our knowledge, the only 
tool for QIF analysis of Java bytecode is our own work jpf-qif p3] which uses SE for 
QIF analysis, but no model counting. Instead jpf-qif adds the conditions for testing 
each bit of the output at the end of the program, hence exploring all these conditions 
using SPF. We compare jpf-qif with QILURA below. 
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We also compare with BitPattern [85], which computes an upper bound on channel 
capacity by exploring the relations between every pair of bits of the output. In 
more recent work [SB], BitPattern was improved using new heuristics. We compare 
QILUR.A with (the improved) BitPattern on several case studies taken from [S3, SB] . 


if 

(H 

> 999){ 

0 

= 

-i; 

> 



0 = 

H; 


0 = 

0 

- H; 


Figure 6.3: No flow 

Moreover, we consider a special case when the program does not leak any information 
to assess the effectiveness and precision of our technique in such a corner case. The 
program does not leak information because the output 0 is always 0 regardless of the 
value of the secret H. However, the assignment O = H and the condition H > 999 
make the program be rejected by other qualitative information-flow techniques, e.g. 
the ones based on type systems or taint analysis. 


6.5.1 Results and discussions 


Figure |6.4| summaries our experiment, we take the time from the faster version of 
BitPattern in [86] . Note that in both [85] and [86] . the authors manually transform 
the programs into bit vector predicates, so there will be extra time if they automate 
this process. 


Case Study 

jpf-qif 

QILURA 

BitPattern 

Capacity 

Time 

Bound 

Time 

Bound 

Time 

No Flow 

0 

2.304 

0 

0.790 

- 

- 

Sanity check, base =0x00001000 

4 

45.324 

4.09 

1.066 

4 

0.036 

Sanity check, base =0x7ffffffa 

4 

35.346 

4.09 

1.049 

4.59 

0.203 

Implicit Flow 

2.81 

0.897 

3 

0.796 

3 

0.011 

Electronic Purse 

2 

1.169 

2.32 

0.854 

2 

0.157 

Ten random outputs 

3.32 

1.050 

3.32 

0.814 

18.645 

0.224 


Figure 6.4: Capacity and bounds are in bits, times are in seconds. means “not 
reported”. 


Comparing with jpf-qif, QILURA has both advantage and disadvantage. As shown 


in Figure 6.4, thanks to the model counting tool Latte, QILURA is much faster than 
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jpf-qif while the upper bounds it computed only deviate to a small extent from the 
exact channel capacities. 

This increase in performance conies with a price, Latte can only count models of 
a system of linear integer inequalities Ax ^ b. For this reason, QILURA cannot 
analyse the case studies of CRC and Tax Record in chapter |4j which have complicated 
constraints. The limitation due to using Latte is shared with previous work [351E5J 
which were also demonstrated with toy examples. 

The BitPattern technique can also compute rather tight upper bounds in most of the 
cases. However, by analysing the relations of pairs of bits, the technique is vulnerable 
when possible values of the output are not in a specific range, as shown in the last 
case study. 

6.6 Discussion of related work 

Self-composition was first introduced by Darvas et al. [50] who expressed it in a 
dynamic logic and proved information flow properties for Java CARD programs. 
Their approach is not automated, requiring users to provide loop invariants, induction 
hypotheses and so on. Barthe et al. [22] then coined the term “self-composition” and 
investigated its theoretical aspects, extending the problem to non-deterministic and 
termination-sensitive cases. 

Terauchi and Aiken []TT3] found that self-composition was problematic, since the self- 
composed programs contains symmetry and redundancy. They proposed a type- 
directed transformation for a simple imperative language to deal with the problem. 
Milushev et al. [57] implemented this type-directed transformation and used Dynamic 
Symbolic Execution (also known as concolic testing ) as a program analysis tool for 
non-interference. 

To our knowledge, our technique is unique in that it only performs analysis on the 
original program, rather than the self-composed program, the idea of self-composition 
is shown in the way we rename the symbolic formula, not in the analysis stage. 

Backes et al. [16] describe how to use the model checker ARMC and Latte for QIF 
analysis. Their technique is very precise but also extremely expensive: it involves 
input counting to compute the pre-image of the observables; in contrast our input 
counting is used for counting the observable. The work of Backes et al. is extended in 
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ra, which uses the interactive theorem prover KeY instead of ARMC, and requiring 
significant user effort. Moreover, this work is based on “ classical” self-composition, 
and as Terauchi and Aiken [TT3] have pointed out, it is unlikely practical. Of course, 
both [[TU] and were demonstrated with toy examples. 

The only technique that can precisely determine if a program leaks information is self¬ 
composition [22]. QILURA also uses self-composition with the key difference that it 
is able to determine if a single symbolic path leaks information. 
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Chapter 7 

A solver for Model Counting 
Modulo Theories 


Recall that in Section 3.3, we have casted the problem of QIF analysis into the 
#SMT problem and proposed two ^SMT-based approaches to QIF, which can be 
summarized by the figure below. 

P <-♦ Tp 


QIF 

Formal methods 


#SMT 

DPLL(T) 


So far we have investigated the approach on the left-hand side: we directly analyse 
the program P, and use formal methods, specifically Bounded Model Checking and 
Symbolic Execution, in a way that mimics a ^SMT solver for QIF analysis. 


This chapter investigates our second approach on the right-hand side: it demonstrates 
how to build from the program P a formula (fip with the two properties described in 


section 3.3, and how to build a t^SMT solver to count the models of ipp. 


Moreover, we study a variant of the #SMT problem: the All-Solution Satisfiability 
Modulo Theories (All-SMT) problem. All-SMT is only different from #SMT in that 
instead of counting the number of model, it asks for the enumeration of all models. 
We show that our algorithms for ^SMT can also be used for All-SMT, and we propose 
the use of an All-SMT solver in new application domains: Bounded Model Checking, 
automated test generation and reliability analysis. 
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7.1 Quantifying Information Leaks using a #SMT 
solver 


We assume the setting in our attacker model in Figure [2TT a program P that takes 
secret input H, public input L and producing public output 0. Our analysis consists 
of two steps as the following. 


• The first step is to build a first-order formula from the program P a formula 
ipp with the following properties: (i) p>p contains a set of Boolean variables 
Vi {pi j P‘2 j ■■jPm}] (h) Pi = T if and only if b t is 1, and p% = - L if and only if 
bi = 0 . 


• The second step is to use a ^SMT solver to count the number of models of ipp 
with respect to the set Vp 

We will show how to build a ^SMT solver later in the next section. At the moment, 
we assume that there is such a solver for our QIF analysis. 


7.1.1 Illustrative Example 


To demonstrate the approach, let us consider again the previously used illustrative 
example in Figure [3J~| which can be encoded into a first-order formula as in Figure ITT} 


L = 8; 


(Li = 8) A 

if (H < 16) 


(G 0 — H 0 < 16) A 

0 = H + L; 


(Oi = H 0 + Li) A 

else 


(O 2 = L\ ) A 

0 = L; 


(O 3 = : 0 3 ) 


Figure 7.1: A simple program encoded into a first-order formula 


The program is transformed into Static Single Assignment (SSA) form [19]: variables 
are renamed when they are reassigned. At the beginning, assuming that the variables 
H, L and 0 take the value H 0 , L 0 and O 0 respectively after being declared. Then, Li 
is the value of the variable L after being reassigned (as 8), and similarly for the rest 
of the program. 


We then need to build the set of boolean variable Vj := {p j, p 2 , ..,p 32 }. In chapter [3j 
we directly analysed the program, and thus hence the set Vj by instrumenting the 
source code of the program P with the block of code in Figure 3.4 This block of 
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code used bitwise operators to extract each bit of the output 0. Here, we analyse the 
formula (encoded from the program), and hence build the set Vj by instrumenting the 
formula. Moreover, the formula needs to be in a first-order theory T that supports 
bitwise operators. Fortunately, the model checker CBMC can automatedly transform 
a C program into a formula in the theory of bit vector OF AUFBV ra which satisfies 
this requirement. 


The formula in Figure 7.1 can be easily expressed in QF_AUFBV with 0 3 declared 
as a 32-bit vector. We instrument the formula by adding a set of Boolean variables 
Vi = {pi,Pi, ■ ■ ■ P 32 }) each one tests the value of a bit of O3. For example: 


(assert (= (= #bl ((_ extract 0 0 ) O 3 )) pi)) 

This statement in SMT-LIB v2 format [21] extracts the first bit of 0 3 (all bits from 
position 0 to position 0), then comparing if this bit is equal to 1 (^bl). The Boolean 
variable p\ is asserted to be the truth value of this comparison. Similar settings are 
applied for the rest of Boolean variables P2,P3, ■ ■ ■ ,P32- 


At this point we have built a formula pp that characterizes the behaviour of the 
program P, and contains a set Vj of Boolean variables, each one represents a bit of 
the output 0 of the program P. By using a ^SMT solver to count the number of 
models of pp with respect to the set V/, which is also the number of possible values 
of the output 0, we can conclude the maximum leakage of the program P as per 
definition El 


7.1.2 Program transformation with CBMC 

In onr two-step analysis, the second step is automated with a #SMT solver, we only 
need to automate the first step: building a formula pp from the program P. 


C(“if(c) A else A”, g) C(h,g A p(c)) A C(/ 2 , g A ~'p(c)) 

■p(“if (c) h else A”, 3 ) := V(h, g A p(c)) A V(I 2 , g A ~>p(c)) 
C(“I 1 ;I 2 ",g):=C(I 1 ,g)AC(I 2 ,g) 

V( u h-,l 2 n ,g) :=V(h,g)AV(I 2 ,g) 

^(“assert (a)”, g) := g —» p(a) 

C(“v = e v ,g) := (v a = (g?p(e) : u a _i)) _ 

Figure 7.2: The two functions C(P, g) and P(P, g) for program transformation. p(c) is 
expression c after being renamed as per SSA form; u Q _i and v a are value of v before 
and after the assignment respectively. 
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The model checker CBMC transforms a program P and a guard g into a logical 
formula using two functions: C(P,g) transforms the program constraints, and V(P,g) 
transforms the program specification, namely assertions. At the beginning, the guard 
g is initialised to T. Both functions are defined by induction on the syntax of program 
as in Figure 7.2 (interested readers are pointed to j?5] for full details). 


In its default settings, CBMC transforms the program and its specification into a 

propositional formula. However, it also has the option-sort2 (still experimental) 

to transform the program and specification into a QF_AUFBV formula in SMT-LIB 
v2 format. Hence, we use leverage this option to build the formula tpp. 


Recall (section 2.4.1) that the formula generated by CBMC is in the form C A -<V, 
where C is the program constraints, and V is the program specification, i.e. assertions. 
Since we only assess the security of a program when it is free from errors, we only 
need the program constraints C. However, without a specification, the aggressive 
program slicing in CBMC can decide immediately that the program does not violate 
any specification, and do not generate any formula. This is actually a strong feature 
of CBMC, however it prevents us from getting a formula C. 


A simple solution for the problem above is to add a fake error, “assert (0) at the 
end of the program. Hence, the generated formula is C A ->_L, or simply C. 


7.1.3 Formula instrumentation 

As we have demonstrated with the example, we need to instrument the formula 
generated by CBMC with a set of Boolean variables Vj. 

In the SSA form, a variable is renamed when it is reassigned. For example, the 
output 0 is named O 0 when initialised. When it is assigned, it is renamed to O i, and 
so on. The index is incremented, and 0 keeps the final value after final assignment. 
Therefore, we build a simple parser to locate the variable Ok renamed from the output 
0, which has the maximal index k. 

The declaration of the set of Boolean variables Vp and theirs binding with the bits of 
Ok can be appended to the end of the formula. The whole formula instrumentation 
procedure is implemented in a simple Java program. 
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7.2 All-Solution Satisfiability Modulo Theories 


The SMT solver MathSAT, from version 4 [3D], provides a functionality, called All- 
SMT, that given a formula ip and a set Vj of important Boolean variables, MathSAT 
in All-SMT mode computes all models of (p with respect to the set Vj. 

In this thesis, we extend the All-SMT of MathSAT with a set Vr of relevant , possibly 
non-Boolean, variables. The extended All-SMT(<y3, V/, Vr) problem is to compute all 
models of (p with respect to the set Vj and the models includes value assignment for 
variables in Vr. We show how this All-SMT problem can be used to analyse the 
availability, reliability and security of programs: 

• Bounded Model Checking [26]: SMT-based Bounded Model Checking can only 
return a single error traces, the user has to fix the error, then run the model 
checker again for other error traces. This is because SMT solvers can only return 
one model. Combining Bounded Model Checking with an All-SMT solver, we 
can compute multiple counterexamples in one run of the model checker. 

• Automated Test Generation: an All-SMT solver can be combined with either 
a Symbolic Executor or a Bounded Model Checker for test input generation. 
Although traditional Symbolic Execution with an SMT solver is capable of 
generating test inputs, it needs to make hundreds or thousands of calls to the 
SMT solver. In our approach, the Symbolic Execution tool needs to make only 
one call to the All-SMT solver for any programs. 

• Reliability analysis: we can build a reliability analysis tool by combining an 
All-SMT solver with a Symbolic Executor to enumerate all path conditions of 
the program, then using the Barvinok model counting technique [5] to compute 
the number of inputs that go into each symbolic path. In this way, we can 
compute the reliability of the program, i.e. the probability that the program 
successfully accomplishes its task without errors. 


7.2.1 Multiple-counterexamples for BMC 


Recall that (section 2.4.1), Bounded Model Checking (BMC) transforms the program 


and its specification into a formula, then solving the resulting formula with a SAT or 
SMT solver. 



Since a SAT or SMT solver can only return a single model, state-of-the-art SAT- 
based or SMT-based Bounded Model Checker can only return a single error trace 
per run. The user has to fix the error and run the model checker again to find more 
error traces. On the other hand, All-SMT solver can return all models w.r.t. a set of 
Boolean variable, it can be exploited to find multiple counterexamples for BMC. 


x=x+y ; 


xi=x 0 +y 0 ; 

if(x !=1){ 


if( Xl !=1){ 

x = 2 ; 


x 2 = 2 ; 

if(z) x++; 

—y 

if(z 0 ) x 3 =x 2 +l ; 

assert (y > 1) ; 


assert (y 0 > 1) ; 

> 


> 

assert (x < 3); 


assert (x 3 < 3) ; 


C ■— xi = x 0 + y 0 A 

x 2 = (On 7^ 1)?2 : a?i) A 
x 3 = (On 7 ^ 1 A z 0 )?x 2 + 1 : x 2 ) 
V := (ari ^ 1 -> y 0 > 1) A (x 3 < 3) 


Figure 7.3: Example, modified from [35]: the program is transformed into Static 
Single Assignment form, and then encoded into a logical formula 


To illustrate our approach, we reconsider an example from [35], which was used to 
illustrate CBMC [33]. The example is shown in Figure 7.3, we have modified it, 
adding the assertion (y > 1), so that the program P contains more than one error. 
At the first step, the program is transformed into SSA form. 


As shown in Figure 7.3, applying C(P, g) and V(P, g) in the SSA program results in 
the set of guards (x\ ^ 1) and (z 0 ^ 0). We denote Boolean variable g 3 and g 2 such 
that gi := BA(xi ^ 1) and g 2 BA(zq ^ 0). 


A model of the formula C A ->V will correspond to a trace of the program that 
violates the specification V. By asking an All-SMT solver to return all models of 
<p = C A -iT* with respect to the set of Boolean abstraction of variables in the guards, 
i.e. Vj = {gi,g- 2 }, we can get a set of all models, each one corresponds to an error 
trace. 


Note that each error trace represents a set of concrete execution, to get the inputs 
for just one representative concrete execution that triggers the error, we set them as 
the relevant variables to be included in the models, which means Vr = {^o, yo, £o}- 
In this case hence Vj represents the guards of the program and Vr the inputs. 
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7.2.2 Automated Test Generation 


This section shows how an All-SMT solver can be used in two different approaches for 
Automated Test Generation (ATG), namely Bounded Model Checking and Symbolic 
Execution. 


ATG using Bounded Model Checking 


We use the same trick as in the previous section. The goal is to compute all models 
of a formula, each one corresponds to a program traces in the program. Take an 
example as in Figure 7.4| Different from the one in Figure [73 the program contains 
no error, so it will go through CBMC without any solver being called. CBMC will 
not generate a formula either. 



( 5-1 = x x > 5) 

A 

(x 2 = 1 + Xi) 

A 

i 92 = x 2 <3) 

A 

(x 3 = -1 + x 2 ) 

A 

to 

A 

( 2/2 = 1 + x 4 ) 

A 

(^5 = : X 4 ) 

A 

(l/3 = 02?2/i : 2 / 2 ) 

A 

(.X'6 = -^gjxi : x 5 ) 

A 

(2/4 = : 2 / 3 ) 



Figure 7.4: A simple program encoded into a formula 


In order to generate test inputs that cover all program paths (to a given bound), 
similar to the previous section, we append “assert (0);” as a fake error at the end 
of the program. Since this error is reachable by all program paths, CBMC will include 
all the paths into the formula. 


The box in the right in Figure 74 shows the formula encoded by CBMC. We run 
the All-SMT solver on the formula with Vj = { 5 - 1 , 5 - 2 }, Vr = {^i,2/i}- The All-SMT 
solver will return a set of solutions, each one contains value assignments for x\ and 
yi, which can be used as test input for the function foo. 
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ATG using Symbolic Execution 


Recall that Symbolic Execution (SE) executes programs on unspecified inputs, by 
using symbolic inputs instead of concrete data. For each executed program path, 
a path condition pc is built which represents the condition on the inputs for the 
execution to follow that path, according to the branching conditions in the code. 
In classical SE, the satisfiability of the path condition is checked at every branching 
point, using off-the-shelf solvers. In this way only feasible program paths are explored. 
Test generation is performed by solving the path conditions. 

Here we propose another approach for SE using All-SMT solver. We use SE with 
the constraint solver turning off, to compute the set of all possible program paths: 
pc\,pc 2 , ■ •. pcm ■ Since there is no constraint solving, a pci can be infeasible. Hence, 
the program under test can be viewed as corresponding to the following formula: 


<p := pci V pc 2 ■ ■ • V pc M 


To illustrate, let us consider again the program in Figure 7.4 The program can be 
viewed as corresponding to the formula: 


<p := ((x > 5) A (x + 1 < 3)) V (7.1) 

((x > 5) A -<(x + 1 < 3)) V 
-i(re > 5) 

Notice that the path (x > 5) A (x + 1 < 3) is infeasible, but it is still included in 
the formula, since we do not check the constraint at each branching point. Applying 
Boolean abstraction on <p leads to: 

BA := ((Gi = (x > 5)) A (C 2 = (x + 1 < 3))) 

We use an All-SMT solver on <pA BA with Vj = [C\, C 2 } and Vr = {x, y}. The set of 
models returned by the All-SMT solver is the set of feasible paths, and the evaluation 
of relevant variables can be used as test inputs for the program. 

Also for ATG Vj represents the guards of the program and Vr the inputs. 


7.2.3 Reliability analysis 

This section introduces an alternative implementation for the approach in [59] by 
using our All-SMT-based SE instead of classical SE. The improvement is that we 
only need to make only one call to the All-SMT solver to explore all feasible paths. 
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void foo(int x, y){ 
if(x > 5) { 
x + +; 

if (x < 3) 
x--; 
else { 

y = x + 1; 
assert false; 

} 

> 

return; 


Let us consider again the previous example with only one difference: we add an error 
for the path x > 3. Similar to the previous section, we use SE with the constraint 


solver turning off, to encode the program into a logical formula p as in (7.1). More¬ 
over, the two paths ((x > 5) A (x + 1 < 3)) and -i(x > 5) are labelled with T. 
as in these two paths the program finishes normally. On the other hand, the path 
((x > 5) A -i(x + 1 < 3)) is labelled with F since an error is reachable in this path. 


Similar to the previous section, using an All-SMT solver on <pABA with V/ = {Ci, C 2 } 
and Vfi = (x, y} will eliminate the infeasible path ((x > 5) A (x + 1 < 3)). We then 
can use the Latte tool to count the models for each paths, and compute the reliability 
of the program. 


7.3 Algorithms for #SMT and All-SMT solver 


Obviously, there is no off-the-shelf solver for our new problem ^SMT. The closet to 
a ^SMT solver is the functionality All-SMT of MathSAT. However, MathSAT does 
not support model generation for relevant variables in All-SMT mode. Moreover, 
when using MathSAT for our analysis, MathSAT returns incorrect number of models 


in several benchmarks (we will show later in section 7.4). For these reasons, we have 


developed a lightweight approach to implement an AII-SMT/t^SMT front-end for 
SMT solvers. We build our algorithms from a number of APIs provided by the SMT 
solver, which we list below. 
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API 


Description 


Assert(^) 

Check() 

Model() 

Eval(t) 

PushQ 

Pop(n= 1) 


Assert formula p into the solver. 

Check consistency of all assertions. 

Get model of the last Check. 

Evaluate expression t in current model. 
Create a backtracking point. 

Backtracks n backtracking points. 


A key feature of SMT solvers for our algorithms is that of being incremental and 
backtrackable. The following example shows a sequence of API calls and their effects 
to the solver. 


Assert (</q); 

CheckQ; 

<Pi 

=>- SAT 

Push(); 


<Pi 


Assert(<yj 2 ); 

CheckQ; 

pi A p 2 

SAT 

Push(); 


pi A p 2 


Assert(<^ 3 ); 

CheckQ; 

Pi A p 2 A p 3 

=>- UNSAT 

Po P (2); 


<Pi 


Assert(<^ 4 ); 

CheckQ; 

Pi A p 4 

=>- SAT 


It is possible for an incremental SMT solver to add additional assertions to the orig¬ 
inal formula. Moreover, when Check is being called several times, the solver can 
remember its computation from one call to the other. Thus, when being called to 
check pi Ap 2 after checking pi, it avoids restarting the computation from scratch by 
restarting the computation from the previous status. Backtrackable means that the 
solver is able to undo steps, using Push and Pop, and returns to a previous status 
on the stack in an efficient manner. 

Both z3 [52] and MathSAT [2D] provide similar APIs to interact with the solver in 
incremental mode. Beside the APIs, we develop a function filter(m, Vj,Vr) that 
given a model of the formula p, and a set of important Boolean variable Vj, and 
the set of relevant variables Vr, the function will return a subset m ir of m that only 
contains literals from Vj and Vr. This function will be used in both algorithms. 


7.3.1 Blocking clauses method 

A straightforward approach for t^SMT is to add clauses that prevent the solver from 
finding the same solution again. 
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function All-BC(<p, Vi, Vr) { 

N G- 0; P G- e; 

Assert(p); 

while (Check() = SAT) { 

N G- N + 1; 

m G- Model(<p); 

m ir filter(m, Vi,Vr); 

G- W U 

block G- FALSE; 
for all pi G Vi do { 

block G- block V (pi ^ Eval(pj)); 
end for} 

Assert (block); 

} 

return N, 1 


Figure 7.5: Blocking clauses #SMT 


The pseudo-code for the blocking clauses method is shown in Figure 7.5 


the solver discovers a solution m of ip such that m = Z 0 A l\ A ■ ■ ■ A l n A .. 
only l 0 , li.. ,l n arc literals of p 0 , pi... p n in Vr The negation of li A l 2 A 
be, by De Morgan’s law, as follows: 


Every time 
., in which 
• A l n would 


block = — iZo V — «Zi V • • • V —>l n 

A literal l t in the model can be viewed as a mapping p t to {TRUE, FALSE}, thus the 
negation -7, isp, ^ Eval(pj). By adding this clause to the formula, by Assert(block), 
a solution with l 0 A l± A ■ ■ ■ A l n will not be discovered again. This procedure repeats 
until no other solution is found. At that point, we have enumerated all the solution 
of (p with respect to Vr All solutions are stored in and N = \\P\ is the result for 
the corresponding ^SMT problem. 

The blocking clauses method is straightforward and it is simple to implement. How¬ 
ever, adding a large number of blocking clauses will consume a large amount of mem¬ 
ory. Moreover, increasing number of clauses also means that the Boolean Constraint 
Propagation procedure is slowed down. Despite these inefficiencies, the blocking 
clauses method can be used to verify the results of other techniques. 


104 




7.3.2 Depth-first search 


To address the inefficiencies of adding a large number of clauses, we introduce an alter¬ 
native method which avoids re-discovering solutions using depth-first search (DFS). 

We divide the set of variables of tp into two sets: Vj is the set of important Boolean 
variables, and Vjj is the set of unimportant, possibly non-Boolean, variables (Vr C 
Vj/). Hence, the formula <p can be viewed as a function: 

Vj X V L j -)■ {TRUE, FALSE} 

Our #SMT procedure is the integration of two components: the first component is 
a simple SAT solver to enumerate all possible partial truth assignments /ij of V}; the 
second component is the SMT solver to check the consistency of A /ij. 


function ALL-DFS(y2, Vj, Vr) { 

N <r- 0; $ <r- e; 

Assert((p); 

if (Check() 7^ SAT) return N, 

depth 0; finished ■<— FALSE: 
while (finished = FALSE) { 
l y- choose_literal(V / j); 
Push(); 

Assert(Z); depth <b- depth + 1; 
if (Check () = SAT) { 
if (depth = |V>|) { 

N <r- N + 1 ; 

m y- Model(tp); 

m ir <- filter(m, Vi,Vr); 

^ ^U {nij r }; 

backtrack(); 

}} 

else backtrack(); 

} 

return M, 


Figure 7.6: Depth-first search #SMT 


The pseudo-code for DFS-based #SMT is depicted in Figure 7.6 The method 


choose_literal chooses the next states to explore from Vj in a DFS manner, and the 
variable depth keeps the number of important variables that has been chosen. That 
means, choose_literal will select a literal Vj-[depth] or -iV}[depth]. This literal is 
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“ pushed ” to the formula as a unit clauses. Recall that the plain DPLL algorithm EU 
is a depth-first search combining with the BCP procedure. Here we do not perform 
BCP, however by adding all literals of /i/ as unit clauses to <p, we force the SMT 
solver to perform BCP on those literals. 

When all important variables has been assigned a truth value, i.e. depth = \Vr\, 
and the formula in the solver is consistent, then the search has found a model. It 
then backtracks to find another one. The method backtrack implements a simple 
chronological backtracking, it “pops” the unit clauses and sets the variable finished 
to TRUE when all states are explored. It is also called when the formula in the solver 
is inconsistent. 

Compare to the blocking clauses method, the DFS-based method is much more effi¬ 
cient in term of memory usage. The blocking clauses method needs to add N blocking 
clauses to find all models while the DFS adds maximum \Vj\ of unit clauses. The 
memory efficiency leads to timing efficiency when there are a large number of models. 

7.3.3 Implementation 

We have implemented both of the methods discussed above in a prototype tool, called 
aZ3. The tool is built in Java, using the APIs provided by the SMT solver z3 [53]. 
aZ3 supports standard SMT-LIB v2 with two additional commands: the first one is 
check-allsat, similar to MathSAT, to specify the list of important variables, and 
the second one is allsat-relevant to specify the list of relevant variables. 

We have also implemented a QIF analyzer, called sqifcj—|-, which uses CBMC to 
encode a program into a formula, then invoking aZ3 to compute channel capacity. 


7.4 Evaluation 

We make two experiments: the first one is to compare our ^SMT solver aZ3 against 
MathSAT modified for ^SMT; the second one is to compare the QIF approach using 
a ^SMT solver with the one in chapter [3j which uses formal methods. 

The benchmarks, the aZ3 solver, and the wrapper of MathSAT for ^SMT can be 
found at: http : //www.eecs.qmul.ac.uk/~qsp30/test/allsmt.tar.gz. 
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7.4.1 Evaluation of ^SMT solvers 


Benchmark 

Expected 

N 

MathS. 

N 

AT 5 

Time 

a 

BC time 

Z3 

DFS time 

QF.LIA 

Example in Figure 

7.3 


2 

2 

0.007 

0.021 

0.013 

Example in Figure 

7.4 


3 

3 

0.005 

0.008 

0.007 

Flap controller [59] 

5 

5 

0.031 

0.020 

0.012 

Red-black tree [][ 

31 

31 

0.016 

0.054 

0.073 

Bubble sort [2] 

541 

541 

0.136 

1.850 

2.069 

Array false [2] 

1370 

1370 

0.037 

3.008 

2.650 

Sum array false [2] 

1024 

1024 

0.026 

0.899 

0.792 

Linear search false [2] 

1024 

1024 

0.028 

0.899 

0.604 

QF.AUFBV 

Data sanitization [85] 

16 

16 

0.008 

0.035 

0.086 

Implicit flow [85] 

7 

7 

0.012 

0.029 

0.049 

Population count [35] 

33 

71 

0.012 

0.074 

0.398 

Mix and duplicate [85] 

65536 

162087 

4.648 

- 

136.947 

Masked copy [85] 

65536 

65536 

1.319 

- 

18.630 

Sum query [85] 

28 

64 

0.010 

0.055 

0.133 

Ten random outputs [85] 

10 

10 

0.014 

0.038 

0.093 

CRC (8) I«n 

8 

12 

0.018 

0.041 

0.099 

CRC (32) [95] 

32 

36 

0.019 

0.075 

0.325 


Figure 7.7: N is the number of models. BC time and DFS time are the time of aZ3 
using the blocking clauses method and depth-first search-based method respectively. 
Times are in seconds. means “timed out in 1 hour”. Notice that for both aZ3 
implementations the number of models is Expected N. 

In order to evaluate aZ3 and MathSAT, we create two set of benchmarks. The 
first group of benchmarks are formulas in QF_LIA (integer linear arithmetic) [21] . 
These benchmarks are used to evaluate All-SMT solvers in the context of test input 
generation. The formulas are generated using Symbolic PathFinder [100] (SPF). SPF 
has a parameter, symbolic.dp, to set the constraint solver for it. If this parameter 
set to no_solver, the tool will run without constraint solving. 

The architecture of SPF enables us to attach a “listener” to it. When SPF executes 
a program, the listener collects the path conditions, and outputs them to a QF LIA 
formula. The models of the integer variables can be used as test inputs for the original 
programs. 

The second group of benchmarks that we considered are formulas in QFAUFBV [2T] 
(bit vector with array). The source of these benchmarks are programs in the QIF 
literature, mostly re-collected in [85]. We use CBMC with the option- smt2 to 
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transform the programs into QF_AUFBV formulas, and instrument the resulting 
formulas to make them ^SMT problems. There are no relevant variables in these 
benchmarks. 


Discussion of evaluation 


Figure 7.7 summaries our experiments with the two solvers aZ3 and MathSAT 5.2.11 
on the benchmarks. In order to compare with MathSAT, we commented out the 
relevant variables in the QF_LIA benchmarks. As shown in the figure MathSAT is 
faster than aZ3. This is not surprised, since we build the tool from the front-end, 
while the All-SMT functionality of MathSAT is built from the back-end, making use 
of the internal data structure. 


However, MathSAT returns incorrect models in several benchmarks. Especially, in 
the benchmark “Mix and duplicate” MathSAT is significantly faster than aZ3, but it 
is also extremely imprecise at the same time. Note that benchmarks in QF_AUFBV 
are derived from the QIF literature, and their number of models were already reported 
in other papers. For example “Mix and duplicate” was reported in [89] and [85] to 
have 2 16 models. 


The blocking clauses methods is comparable, or even faster than the DFS-based 
method when the number of models is small. However, for the benchmarks with 
2 16 models, adding 2 16 blocking clauses is obviously not efficient in both time and 
memory. As a result, the method failed to provide the answer for such benchmarks. 
On the other hand, the DFS-based method was still able to provide the answer in a 
reasonable time. 


7.4.2 Evaluation of QIF analysers 


Figure A8 compares the performance of sqifc++ against the tool sqifc in chapter [3] 
The results show that sqifc++ is much more efficient. The reason is that, sqifc makes 
several calls to CBMC, and for each call CBMC has to transform the program into 
a formula, then calling a SAT/SMT solver to check the formula. On the other hand, 
sqifc++ transform the program only once, and its search is also more efficient. 


However, sqifc++ relies on CBMC for program transformation, and this functionality 
of CBMC (-smt2) is still experimental. We found that CBMC generates incorrect 
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Benchmark 

Leaks 

sqifc 

sqifc++ time 

Total 

time 

CBMC time 

aZ3 time 

Data sanitization [85| 

4 

11.898 

0.165 

0.086 

0.251 

Implicit flow [83] 

2.81 

5.033 

0.169 

0.049 

0.218 

Population count [SB] 

5.04 

17.278 

0.162 

0.398 

0.560 

Mix and duplicate [85] 

16 

- 

0.154 

136.947 

137.101 

Masked copy [85] 

16 

- 

0.175 

18.630 

18.805 

Sum query [85 

4.81 

64.557 

0.162 

0.133 

0.295 

Ten random outputs [83] 

3.32 

64.202 

0.160 

0.093 

0.253 

CRC (8) [93] 

3 

2.551 

0.184 

0.099 

0.283 

CRC (32) [93] 

5 

7.755 

0.193 

0.325 

0.518 


Figure 7.8: Comparing the new approach with the sqifc tool in chapter [3} Leaks are 
in bits. aZ3 runs with the DFS-based algorithm. Times are in seconds, means 
timeout in one hour. Total time of sqifc++ is the sum of CBMC time and aZ3 time. 


formulas for several case studies in chapter [3j therefore we could not use sqifc++ to 
analyse, for example, the dining cryptos case study. 


7.5 Discussion of related work 

7.5.1 Quantitative Information Flow 

We have just compared the two prototype tools sqifc++ and sqifc in the previous 
section. Moreover, in chapter [3j we already compared our ^SMT-based approach 
with other work in QIF literature, so we do not repeat the discussion here. 

7.5.2 Multiple-counterexamples for BMC 

The most relevant work to ours is that of Bhargavan et al. [24] embodied in the 
Verisim testing tool for network protocols. When an error trace is found to violate 
the specification, which is an extended LTL formula 0, Verisim uses a technique, 
called tuning , to replace (j) with tp that ignores the violation. Tuning is not fully 
automatic. 

Another technique introduced by Ball et al. [18] is embodied in the SLAM tool-kit. 
The algorithm uses a model checker as a sub-routine. When the model checker finds 
an error trace, SLAM localizes the error cause, modifying the source code with a halt 
statement at the error cause. The model checker is then invoked again, and the halt 
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statements instruct the model checker to stop exploring paths at the previously found 
error causes. This procedure is very expensive, it requires comparing the error trace 
with all correct traces to localize the error, and requires to run the model checker 
several times. Our work is much simpler, and faster but the error traces we compute 
can come from the same causes. 

7.5.3 Automated Test Generation 

The closest to our work is FShell [57] , which also uses CBMC for automated test 
generation. FShell transforms the program under test into a CNF formula, and 
solves it using an incremental SAT solver. Every time the SAT solver finds a solution 
representing a symbolic path, FShell adds a blocking clauses to prevent that path 
from being explored again. As our experiments have shown, the blocking clauses 
method is suffered from rapid space growth. 

Classical Symbolic Execution also uses SMT solvers to check the satisfiability of path 
condition. The SMT solver is called whenever a conditional statement is executed, 
hence it may be called hundreds or thousands of times. In our approach, the symbolic 
executor makes only one call to the All-SMT solver. 

7.5.4 Reliability analysis 

Our approach to reliability analysis is based on the paper of Filieri et al. [50] that 
uses classical Symbolic Execution and Barvinok model counting tool. We extend 
the approach using our new All-SMT-based Symbolic Execution instead of classical 
Symbolic Execution. The main difference is the same as in the case of test generation, 
our approach only makes one call to the All-SMT solver in the whole analysis. 

7.5.5 All Solutions SAT Modulo Theories 

As we have discussed throughout the chapter, MathSAT is the only SMT solver 
that supports All-SMT. Its algorithm is briefly described in [02], and it has been 
used to compute predicate abstraction in [33j and fUJ. Our experiment results show 
that MathSAT is imprecise in several benchmarks. However, we also notice that 
the imprecisions seem to be limited in QF_AUFBV benchmarks, while the authors 
performed experiments with QF_LRA formulas in [34]. 
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Also in the context of predicate abstraction, Laliiri et al. CZ2] have proposed several 
techniques, which use the SMT solver Barcelogic to generate the set of all satisfying 
assignments over a set of predicates. However, we are not able to include Barcelogic 
in our experiments, since the solver provided to us by the author does not support 
AU-SMT. 

A principal difference between the work mentioned above and the one in this chapter 
is that we implemented from the front-end of an SMT solver. For this reason, our 
implementation is slower than MathSAT. On the good side, our approach is applicable 
to implement even in closed-source SMT solvers that do not support All-SMT but 
provide similar APIs. 


Ill 


Chapter 8 


Conclusions 


8.1 Summary 

This thesis introduces a new research problem, Model Counting Modulo Theories 
or ^SMT, and presenting a ^SMT-based approach to quantification of information 
leaks. Although our implementations are far from being optimised, they drastically 
outperform the existing technique based on self-composition, e.g. reducing the time 
of analysing some programs from Linux kernel from some hours to a few seconds. Our 
approach is applicable to programs with difficult data structures including pointers, 
and to Java bytecode. 

On the theoretical side, this thesis makes the original contributions by discovering 
the relations among different research areas: (i) it casts the QIF problem into the 
^SMT problem; (ii) it shows the correspondence between Symbolic Execution and 
the DPLL(T) algorithm; (iii) it explores the relation between Symbolic Execution 
and Bounded Model Checking; (iv) finally, it exploits the connection between QIF 
analysis and Reliability analysis. 

On the application side, this thesis is the first to use Symbolic Execution to quantify 
information leaks. It is also the first to use classical Symbolic Execution for Bounded 
Model Checking. Beside, it proposes the use of an All-SMT solver for multiple- 
counterexamples in Bounded Model Checking, for automated test generation, and for 
reliability analysis. 

On the practical side, this thesis has developed several tools in both C/C++ and Java: 
sqifc, jpf-qif and QILLIRA, for the quantification of information leaks in software. It 
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also demonstrated the use of these tools to analyse vulnerabilities from the National 
Vulnerability Database of the US government, and anonymity protocols. 

For the verification community, important contributions of this thesis include the 
development of JCBMC, a Concurrent Bounded Model Checker for Java, and the 
All-SMT solver aZ3. 


8.2 Future Research 

In the previous chapter, we have proposed the use of an All-SMT solver for multiple- 
counterexamples in Bounded Model Checking, for automated test generation and for 
reliability analysis. An immediate direction would be to implement these ideas into 
automated tools, and to perform experiments on standard benchmarks. Some other 
possible directions for investigation are the following. 

Fault localization 

Another interesting avenue of further research would be to the All-SMT solver with 
CBMC to localize error causes using similar idea in [TKj. Models of = C A V 
correspond to correct traces that satisfy the specification, and models of = C A -SP 
correspond to error traces that violate the specification. Using an All-SMT solver, 
we can compute the sets of all models of ipi and tp 2 with respect to the set of guards. 
Comparing the two sets of models, we can localize the transitions that only appear 
in error traces. 

Concurrent Bounded Model Checking 

A first improvement on this direction would be to upgrade its concurrency from 
single CPU multi-threading to true parallelism and to perform obvious optimisations. 
Another improvement would be to replace Symbolic PathFinder with a lighter weight 
tool, or a parallel version, to reduce the cost of generating path conditions. It will 
also be interesting to implement the methodology for other languages (C, Python) 
and investigate how to use Symbolic Execution for IC3 style verification. 
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Statistical analysis for QIF 


The QILURA tool is still just a prototype. A possible improvement for it would be 
to use approximate exploration techniques to replace the exact, complete exploration 
presented in this thesis. In this way, for the tool can be used with increased scalability, 
but with formal statistical guarantees on the results. 
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